Construction method and application of Hash code generation model of remote sensing image

Abstract

The invention discloses a remote sensing image hash code generation model construction method and application, and belongs to the technical field of image retrieval. Firstly, a target detection module is used for detecting a potential target in a remote sensing image, and introduction of invalid information can be reduced as much as possible during feature extraction; the target is mapped to a unified hash space through a hash module, so that an accurate hash code can be generated; meanwhile, the model is obtained through unsupervised learning training, information loss caused by discretization of continuous hash codes by the hash code generation model is reduced as much as possible by minimizing quantization loss, manual annotation data are not needed, training can be performed on a large-scale unlabeled remote sensing image data set, and the method is high in robustness and high in robustness. The problem that sufficient labels are difficult to obtain for large-scale data in practical application is solved, and accurate hash codes can be generated for remote sensing images at low time cost and economic cost.

Description

Technical Field

[0001] The present invention belongs to the technical field of image retrieval, and more specifically, relates to a method for constructing a hash code generation model for remote sensing images and its application. Background Art

[0002] With the development of earth observation technology, the number of remote sensing images has shown an explosive growth trend. How to retrieve target images from a large number of remote sensing databases has become a current research hotspot. The content-based image retrieval method (CBIR) starts from understanding the features of the image itself, uses the content features of the image as an index to construct an image database, and then realizes retrieval. Compared with the traditional keyword matching-based retrieval method, content-based retrieval can better meet the needs of modern remote sensing image retrieval. As a branch of CBIR, the image hashing method has become one of the popular solutions in the field of image retrieval because it can compress high-dimensional features into binary codes, reducing memory occupancy while improving retrieval efficiency.

[0003] Existing image hashing methods often default that there is significant target object information in the image and directly perform semantic extraction on the scale of the complete image. However, in remote sensing images, the scale gap between the target and the image is too large, and the target image often only occupies a very small part of the complete image and is randomly distributed at different positions in the image. A large amount of effective information will be lost according to the traditional processing method, and accurate hash codes cannot be generated for the image. At the same time, the types, quantities, and location information of remote sensing image targets are all unknown. For remote sensing image datasets, the target annotation process highly depends on manual labor and professional knowledge, requiring a large amount of time and economic costs, which poses an obstacle to the realization of automated model construction. Summary of the Invention

[0004] Aiming at the above defects or improvement requirements of the prior art, the present invention provides a method for constructing a hash code generation model for remote sensing images and its application, so as to solve the technical problem that the prior art cannot generate accurate hash codes for remote sensing images at a low time cost and economic cost.

[0005] To achieve the above object, in a first aspect, the present invention provides a method for constructing a hash code generation model for remote sensing images, including:

[0006] Train the hash code generation model in one or more batches to obtain a trained hash code generation model; the hash code generation model includes: a target detection module for performing target detection on a remote sensing image to obtain the position information of each target in the remote sensing image, and extracting the targets in the remote sensing image from the remote sensing image to obtain the target sub-images included in the remote sensing image; a feature extraction module for extracting the features of the target sub-images; a hash module for converting the features of the target sub-images into corresponding continuous hash codes; a mapping module for binarizing and mapping the continuous hash codes of the target sub-images into corresponding discrete hash codes; the hash code of the remote sensing image includes the discrete hash codes of its target sub-images;

[0007] Wherein, the following operations are performed in each training batch:

[0008] Input the remote sensing images in the current training batch into the hash code generation model to obtain the continuous hash codes and discrete hash codes of each target sub-image in each remote sensing image;

[0009] Construct a training objective including minimizing the total quantization loss, and train the hash code generation model based on the training objective; wherein, the total quantization loss is the sum of the quantization losses of all target sub-images in the current training batch; the quantization loss of the target sub-image is used to measure the difference degree between the continuous hash code and the discrete hash code of the target sub-image.

[0010] Further preferably, the target detection module also obtains the category information of each target in the remote sensing image when performing target detection on the remote sensing image; the category information includes: the category and confidence of the target;

[0011] The target sub-images obtained by the above target detection module are target sub-images with a confidence greater than or equal to a preset threshold;

[0012] The hash code of the remote sensing image includes the discrete hash codes of the target sub-images with an in-built confidence greater than or equal to the preset threshold.

[0013] Further preferably, the above training objective also includes: minimizing the positive difference loss and maximizing the negative difference loss

[0014] Wherein, bs is the number of target sub-images in the previous training batch; n(P i ) is the number of target sub-images in the positive sample set P i of the i-th target sub-image; P i is the set composed of other target sub-images with the same category as the i-th target sub-image except the i-th target sub-image among all the target sub-images in the current training batch; w j is the confidence corresponding to the j-th target sub-image in P i ; n(Ni ) is the negative sample set N of the i-th target subgraph i ; the number of target subgraphs in loss ij is the continuous hash code H of the i-th target subgraph i and P i the continuous hash code H of the j-th target subgraph in j '; the difference loss between n(N i ) is the set composed of target subgraphs with different categories from the i-th target subgraph among all target subgraphs in the current training batch; w k is N i the confidence corresponding to the k-th target subgraph in; loss ik is the continuous hash code H of the i-th target subgraph i and N i the continuous hash code H of the k-th target subgraph in k '; the difference loss between

[0015] Further preferably, minimizing the positive difference loss is achieved by minimizing the contrast loss and maximizing the negative difference loss where the contrast loss is: is the similarity between the continuous hash code H i and the continuous hash code H j '; is the similarity between the continuous hash code H i and the continuous hash code H k '; the greater the similarity of the hash code, the smaller the corresponding difference loss; γ is a hyperparameter; D(H i , H j ') is the codeword distance between the continuous hash code H i and the continuous hash code H j '; D(H i , H k ') is the codeword distance between the continuous hash code H i and the continuous hash code H k '.

[0016] Further preferably, the above mapping module uses the sign function to binary map the continuous hash code of the target subgraph to the corresponding discrete hash code; the total quantization loss is:

[0017]

[0018] where bs is the number of target subgraphs in the previous training batch; L i is the length of the continuous hash code of the i-th target subgraph generated by the hash code generation model; cosh(·) is the hyperbolic cosine function; H i,lIt is the value of the l-th dimension in the consecutive hash code of the i-th target subgraph.

[0019] Further preferably, the method for constructing the hash code generation model of the remote sensing image further includes: after completing the training of the hash code generation model, retaining the discrete hash codes of each target subgraph in all the remote sensing images used for training; using the discrete hash codes of the target subgraphs as hash nodes, the target subgraphs as target nodes, and the remote sensing images as image nodes, connecting the hash nodes with a codeword distance less than or equal to the preset codeword distance, connecting the target nodes with the hash nodes corresponding to their discrete hash codes, and connecting the image nodes with the target nodes corresponding to each target subgraph contained therein to construct a hash graph.

[0020] In a second aspect, the present invention provides a method for generating a hash code of a remote sensing image, including: inputting the remote sensing image to be generated with a hash code into the hash code generation model to obtain the hash code of the remote sensing image;

[0021] Wherein, the hash code generation model is constructed by using the method for constructing the hash code generation model provided in the first aspect of the present invention.

[0022] In a third aspect, the present invention provides a method for hash retrieval of a remote sensing image, including:

[0023] S1. Input the single-target image to be retrieved into the hash code generation model to obtain the corresponding discrete hash code as the target hash code;

[0024] S2. Determine whether the target hash code exists in the hash graph. If not, go to S3; otherwise, go to S4;

[0025] S3. Add the target hash code as a new hash node to the hash graph and connect it with the remaining hash nodes in the hash graph whose codeword distance from the target hash code is less than or equal to the preset codeword distance;

[0026] S4. In the hash graph, starting from the target hash code as the initial hash node, find the neighboring hash nodes within a preset number of steps, and select the K neighboring hash nodes with the smallest codeword distance from the target hash code as candidate hash nodes; K≥1;

[0027] S5. In the hash graph, find the target nodes connected to the candidate hash nodes as candidate target nodes; find the image nodes connected to the candidate target nodes as candidate image nodes; count the number of candidate target nodes connected to each found candidate image node, and sort the candidate image nodes in descending order according to the number of candidate target nodes connected thereto, and output them as the retrieval result;

[0028] Among them, the hash code generation model and the hash map set are obtained by using the hash code generation model construction method provided in the first aspect of the present invention.

[0029] In a fourth aspect, the present invention provides an electronic device, including: a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it executes the method provided in the first aspect, the second aspect or the third aspect of the present invention.

[0030] In a fifth aspect, the present invention further provides a computer-readable storage medium, the computer-readable storage medium includes a stored computer program, wherein when the computer program is run by a processor, it controls the device where the storage medium is located to execute the method provided in the first aspect, the second aspect or the third aspect of the present invention.

[0031] In a sixth aspect, the invention further provides a computer program product, including a computer program / instructions, and when the computer program / instructions are executed by a processor, the method provided in the first aspect, the second aspect or the third aspect of the present invention is implemented.

[0032] Generally speaking, through the above technical solutions conceived by the present invention, the following beneficial effects can be achieved:

[0033] 1. The present invention provides a method for constructing a hash code generation model for remote sensing images. First, a target detection module is used to detect potential targets in the remote sensing image, which can minimize the introduction of invalid information during feature extraction; then, a hash module is used to map the targets to a unified hash space, which can generate accurate hash codes; at the same time, the model in the present invention is trained through unsupervised learning, and by minimizing the quantization loss, the information loss caused by discretizing the continuous hash code by the hash code generation model is minimized as much as possible. It does not rely on manually labeled data and can be trained on a large-scale unlabeled remote sensing image dataset, solving the problem that it is difficult to obtain sufficient labels for large-scale data in practical applications, and can generate accurate hash codes for remote sensing images at a low time cost and economic cost.

[0034] 2. Further, in the method for constructing a hash code generation model provided by the present invention, by using the category and confidence information obtained by target detection, during the training process, only the target subgraphs with a confidence greater than or equal to a preset threshold are retained, further reducing the introduction of invalid information and further improving the accuracy of the model.

[0035] 3. Further, in the method for constructing a hash code generation model provided by the present invention, during the training process, the above training objective further includes: minimizing the positive difference loss and maximizing the negative difference loss The codeword distance between consecutive hash codes of target subgraphs of the same category is shortened, and the codeword distance between consecutive hash codes of target subgraphs of different categories is increased, so that hash codes with similar semantics are mapped to adjacent positions in a unified codeword space, thereby giving the hash module the ability to express semantic similarity and further improving the accuracy of the model.

[0036] 4. Furthermore, the hash code generation model construction method provided by the present invention also includes, after the training is completed, using the discrete hash code of the target subgraph as the hash node, the target subgraph as the target node, and the remote sensing image as the image node, connecting the hash nodes whose codeword distance is less than or equal to the preset codeword distance, connecting the target node with the hash node corresponding to its discrete hash code, and connecting the image node with the target nodes corresponding to each target subgraph it contains, constructing a hash graph, establishing a "remote sensing image-target subgraph-hash code" mapping relationship that can persistently maintain the similarity relationship between targets, and can aggregate data with similar content to adjacent positions in the graph, can avoid traversal search in the subsequent search process and repetitive calculations during multiple retrievals, and effectively improve the efficiency of multi-target remote sensing image retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram of a flow chart of a method for constructing a hash code generation model provided by an embodiment of the present invention;

[0038] Figure 2 A flow chart of a hash retrieval method for remote sensing images provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0040] In order to achieve the above-mentioned object, in a first aspect, the present invention provides a method for constructing a hash code generation model of a remote sensing image, comprising:

[0041] Train the hash code generation model in one or more batches to obtain a trained hash code generation model; the hash code generation model includes: a target detection module for performing target detection on a remote sensing image to obtain the position information of each target in the remote sensing image, and extracting the targets in the remote sensing image from the remote sensing image to obtain the target sub-images included in the remote sensing image; a feature extraction module for extracting the features of the target sub-images; a hash module for converting the features of the target sub-images into corresponding continuous hash codes; a mapping module for binarizing the continuous hash codes of the target sub-images into corresponding discrete hash codes; the hash code of the remote sensing image includes the discrete hash codes of its target sub-images;

[0042] Wherein, the following operations are performed in each training batch:

[0043] Input the remote sensing images in the current training batch into the hash code generation model to obtain the continuous hash codes and discrete hash codes of each target sub-image in each remote sensing image;

[0044] Construct a training objective including minimizing the total quantization loss, and train the hash code generation model based on the training objective; wherein, the total quantization loss is the sum of the quantization losses of all target sub-images in the current training batch; the quantization loss of the target sub-image is used to measure the degree of difference between the continuous hash code and the discrete hash code of the target sub-image.

[0045] It should be noted that the target detection module can adopt any existing target detection module, such as VGG, ResNet, DenseNet, Transformer, etc., which are not limited here.

[0046] The feature extraction module can adopt any existing feature extraction module, such as CNN, ResNet, Transformer, etc., which are not limited here.

[0047] It should be noted that there are various ways to measure the degree of difference between the continuous hash code and the discrete hash code of the target sub-image, such as Euclidean distance, cosine similarity, Manhattan distance, etc., which are not limited here.

[0048] It should be noted that the hash module can be CNN, artificial neural network, autoencoder, graph neural network, etc., which are not limited here.

[0049] In an alternative embodiment, the target detection module also obtains the category information of each target in the remote sensing image when performing target detection on the remote sensing image; the category information includes: the category and confidence of the target;

[0050] The target sub-images obtained by the above target detection module are target sub-images with a confidence greater than or equal to a preset threshold;

[0051] The hash code of the remote sensing image includes the discrete hash codes of the target subgraphs whose built-in reliability is greater than or equal to the preset threshold.

[0052] In an alternative implementation, the above training objective further includes: minimizing the positive difference loss and maximizing the negative difference loss

[0053] where bs is the number of target subgraphs in the previous training batch; n(P i ) is the number of target subgraphs in the positive sample set P i of the i-th target subgraph; P i is the set composed of other target subgraphs with the same category as the i-th target subgraph among all target subgraphs in the current training batch except the i-th target subgraph; w j is the confidence corresponding to the j-th target subgraph in P i ; n(N i ) is the number of target subgraphs in the negative sample set N i of the i-th target subgraph; loss ij is the difference loss between the continuous hash code H i of the i-th target subgraph and the continuous hash code H i ' of the j-th target subgraph in P j ; n(N i ) is the set composed of target subgraphs with different categories from the i-th target subgraph among all target subgraphs in the current training batch; w k is the confidence corresponding to the k-th target subgraph in N i ; loss ik is the difference loss between the continuous hash code H i of the i-th target subgraph and the continuous hash code H i ' of the k-th target subgraph in N k .

[0054] In an alternative implementation, minimizing the positive difference loss is achieved by minimizing the contrast loss and maximizing the negative difference loss where the contrast loss is: is the similarity between the continuous hash code H i and the continuous hash code H j '; is the similarity between the continuous hash code H i and the continuous hash code H k '; the greater the similarity of the hash codes, the smaller the corresponding difference loss; γ is a hyperparameter; D(H i ,H j ) is the continuous hash code H iThe codeword distance between the continuous hash code H j ' is; D(H i , H k ') is the codeword distance between the continuous hash code H i and the continuous hash code H k '.

[0055] It should be noted that there are various ways to measure the codeword distance, and it can be measured by methods such as Euclidean distance, cosine similarity, Manhattan distance, etc., which are not limited here.

[0056] It should be noted that the above mapping module can use functions such as the sign function and the sigmoid function to binary map the continuous hash code of the target subgraph to the corresponding discrete hash code, which is not limited here.

[0057] Preferably, in an alternative embodiment, the above mapping module uses the sign function to binary map the continuous hash code of the target subgraph to the corresponding discrete hash code; the total quantization loss is:

[0058]

[0059] where bs is the number of target subgraphs in the previous training batch; L i is the length of the continuous hash code of the i-th target subgraph generated by the hash code generation model; cosh(·) is the hyperbolic cosine function; H i,l is the value of the l-th dimension in the continuous hash code of the i-th target subgraph.

[0060] In an alternative embodiment, the method for constructing the hash code generation model of the above remote sensing image further includes: after the training of the hash code generation model is completed, retaining the discrete hash code of each target subgraph in all the remote sensing images used for training; using the discrete hash code of the target subgraph as the hash node, using the target subgraph as the target node, using the remote sensing image as the image node, connecting the hash nodes with a codeword distance less than or equal to the preset codeword distance, connecting the target node with the hash node corresponding to its discrete hash code, and connecting the image node with the target nodes corresponding to each target subgraph it contains to construct a hash map.

[0061] To further illustrate the method for constructing the hash code generation model of the remote sensing image provided by the present invention, the following will be described in detail with a specific embodiment:

[0062] In order to improve the accuracy of multi-object remote sensing image retrieval, the present invention provides a multi-object remote sensing image retrieval method based on similarity hashing. The overall concept is as follows: During the training process of the hash code generation model, first use the object detection module to identify potential objects in the original image, obtain the corresponding position information, and extract features from the corresponding object image regions to minimize the introduction of invalid information. On this basis, construct a contrast loss to guide the model to update parameters. In addition, construct a hash graph with the hash codes generated by the hash code generation model as nodes, and establish a mapping relationship of "remote sensing image - object - hash code" to achieve image retrieval oriented to object content similarity and improve the retrieval efficiency of multi-object remote sensing images.

[0063] Based on the above concept, the construction method of the hash code generation model in this embodiment can be mainly divided into two stages, namely the training of the hash code generation model and the construction of the hash graph. The specific details of each stage are as follows:

[0064] The first stage: Training of the hash code generation model:

[0065] Use the pre-trained object detection module to process the original remote sensing image to obtain the potential object information therein, specifically including the four vertex coordinates of the bounding box, the potential category, and the confidence. In order to improve the quality of training samples, select the objects with a confidence above a certain threshold, and intercept the corresponding positions of the objects according to the coordinates to obtain the object sub-images contained in the remote sensing image, and input them into the pre-trained feature extraction module to output high-dimensional feature vectors, that is, the feature vectors of each object sub-image.

[0066] For each object sub-image in the current training batch, discriminate its category information from other object sub-images. Object sub-images with the same category are regarded as positive samples, otherwise they are regarded as negative samples. Thus, a corresponding positive sample set and negative sample set are constructed for each object sub-image. Input the feature vectors of the object sub-images extracted by the feature extraction module into the hash module to obtain continuous hash codes. For the continuous hash codes corresponding to each object sub-image, calculate the codeword distances between them and the continuous hash codes corresponding to the samples in the positive and negative sample sets respectively, and calculate the weighted sum with the confidence as the weight. Finally, obtain the contrast loss. Calculate the numerical change of each continuous hash code after discretization (i.e., binary mapping) to obtain the quantization loss. After obtaining the quantization loss, update the parameters of the hash code generation model in multiple rounds according to the backpropagation rule until the hash code generation model converges or the training terminates.

[0067] The codeword distance in this embodiment is calculated in the following way:

[0068] For any two continuous hash codes, their similarity is determined by the Hamming distance. Since the output at this time is a real value and the similarity cannot be calculated by XOR, an approximate calculation of the Hamming distance is adopted. It is defined as follows:

[0069]

[0070] Among them, H1H2 represents the inner product of two consecutive hash codes H1 and H2, ||·|| is the two-norm of the consecutive hash codes; H1, H2 ∈ [-1, 1] 1×L , and L is the length of the hash code.

[0071] The hash map construction stage includes:

[0072] Using the trained hash code generation model, discrete hash codes are generated for each extracted target subgraph, and the codeword distances between pairwise discrete hash codes are calculated respectively. The hash code generation model obtained through training has the ability to express semantic similarity, so that the discrete hash codes corresponding to target subgraphs with similar content are also close. For each discrete hash code, the similar relationships with codeword distances within a specific threshold are screened to construct an undirected weighted graph with discrete hash codes as nodes, similar relationships as edges, and codeword distances as weights, which can place similar discrete hash codes in neighboring positions in the graph. In addition, a mapping of "image - target - hash code" is constructed, and by inputting the retrieval target, similar targets and the original images can be quickly retrieved. In this embodiment, the codeword distance is the Hamming distance.

[0073] In this embodiment, the above contrast loss and quantization are combined to obtain the overall loss function in the training process of the hash code generation model, and the hash code generation model is trained based on the overall loss function until the loss function converges. During the training process of the hash code generation model, through the constraint of the contrast loss, hash codes with similar semantics can be mapped to neighboring positions in a unified codeword space (the Hamming space in this embodiment); through the quantization loss, the continuous hash codes obtained by the hash code generation model can be constrained to be as close as possible to the discretized results, reducing the final information loss. In addition, the construction of the hash map can persistently maintain the similarity relationship between targets, avoiding repeated calculations during multiple retrievals. Finally, the hash map constructed based on the extracted hash codes can effectively improve the efficiency of multi-target remote sensing image retrieval.

[0074] Such as Figure 1 is the specific process of the construction method of the hash code generation model provided in this embodiment, including:

[0075] Perform the following operations in each training batch:

[0076] 1) First, the target detection module is used to extract the location and category information of potential targets in the original remote sensing image in the current training batch, where the location information is the pixel coordinates of the target in the image, and the category information is the category and confidence predefined by the target positioning algorithm, which is used to guide the learning of the subsequent hash module. The target detection module inputs the image area corresponding to the target, that is, the target sub-image in the remote sensing image, into the pre-trained feature extraction module to obtain continuous vector features, that is, the feature vector of the target sub-image.

[0077] Optionally, in this embodiment, the target detection module is a pre-trained PP-YOLOE-R model, and the output results include the target rotation box coordinates, target confidence, and one of the 15 target categories. The target subgraph with a confidence not less than a preset threshold (the value is 0.3 in this embodiment) is selected as a training sample. The feature extraction module is a pre-trained VGGNet, and the last softmax layer of the original model is removed to obtain a 4096-dimensional vector as the feature vector of the target subgraph.

[0078] It should be noted that this is only an optional implementation of the present invention and should not be understood as the sole limitation of the present invention. In some other embodiments of the present invention, other target detection modules and feature extraction modules may also be used, and the subsequent hash module for converting the feature vector of the target subgraph into a continuous hash code may also be implemented using other network structures that can achieve the same function.

[0079] 2) Use the hash module to transform the feature vector of each target subgraph in the current training batch to obtain a continuous hash code. And according to the category information output by the target detection module, construct a comparative learning positive and negative sample set for each target subgraph in the batch, where the same category is the positive sample set, and the different category is the negative sample set.

[0080] 3) Construct a loss function, including contrast loss and quantization loss. Contrastive loss calculates the codeword distance between the continuous hash code of each target subgraph in the current training batch and other continuous hash codes as a similarity measure, and sums them up with the confidence of the target detection module as a weight. Contrastive loss can shorten the codeword distance between continuous hash codes of target subgraphs of the same category, and lengthen the codeword distance between continuous hash codes of target subgraphs of different categories, thereby giving the hash module the ability to express semantic similarity. Quantization loss minimizes the information loss caused by the discretization of continuous hash codes by the hash code generation model.

[0081] In this embodiment, the target position type detected by the target detection module is a rotating box, which is expressed by four vertex pixel coordinates, namely (x1, y1), (x2, y2), (x3, y3), (x4, y4). When extracting target features, it is considered that the boundaries of the target image area are

[0082] up = min{x1, x2, x3, x4}

[0083] down = max{x1, x2, x3, x4}

[0084] left = min{y1, y2, y3, y4}

[0085] right = max{y1, y2, y3, y4}

[0086] Among them, up, down, left, and right are the upper, lower, left, and right boundaries of the target area, respectively.

[0087] The expression of the contrast loss is as follows:

[0088]

[0089] Among them, bs is the number of target sub - graphs in the previous training batch; n(P i ) is the number of target sub - graphs in the positive sample set P of the i - th target sub - graph; P i is the set composed of other target sub - graphs with the same category as the i - th target sub - graph except the i - th target sub - graph among all target sub - graphs in the current training batch; w i is the confidence corresponding to the j - th target sub - graph in P j ; n(N i ) is the number of target sub - graphs in the negative sample set N of the i - th target sub - graph; n(N i ) is the set composed of target sub - graphs with different categories from the i - th target sub - graph among all target sub - graphs in the current training batch; w i is the confidence corresponding to the k - th target sub - graph in N i ; D(H k , H i ') is the codeword distance between the continuous hash code H i and the continuous hash code H j '; γ is a hyper - parameter; D(H i , H j ') is the codeword distance between the continuous hash code H i and the continuous hash code H k '. i and the continuous hash code H k '.

[0090] The expression of the quantization loss is as follows:

[0091]

[0092] Among them, bs is the number of target sub - graphs in the previous training batch; L iis the length of the consecutive hash code of the i-th target subgraph generated by the hash code generation model; cosh(·) is the hyperbolic cosine function; H i,l is the value of the l-th dimension in the consecutive hash code of the i-th target subgraph.

[0093] The expression of the total loss function is as follows:

[0094] L = λL c + L Q

[0095] where λ is a trade-off parameter.

[0096] If the total loss function does not converge, continue training for the next training batch; otherwise, obtain the trained hash code generation model, fix the parameters in the current hash code generation model, and use it for subsequent hash code generation.

[0097] Generally speaking, in this embodiment, the target detection module is first used to locate the potential target position information in the remote sensing image, and feature extraction is performed on the target subgraph (i.e., the target area) in the remote sensing image, fully focusing on the effective information and reducing the introduction of noise information; on the basis of the target information, the contrast learning framework is used to help the model learn a unified Hamming space, in which the hash codes with similar semantics can be closely mapped together, effectively enhancing the semantic expression ability of the model. Finally, the hash codes extracted by the trained hash code generation model can effectively improve the accuracy of multi-target remote sensing image retrieval.

[0098] The second stage: construction of the hash graph

[0099] For the data set composed of remote sensing images containing multiple targets, the discrete hash codes of the target subgraphs in each remote sensing image are extracted by using the above hash code generation model, and the number of graph edges to be retained is limited by the threshold T. Specifically, after the training of the hash code generation model is completed, the hash codes of each target subgraph in all the remote sensing images used for training are retained; using the discrete hash codes of the target subgraphs as hash nodes, the target subgraphs as target nodes, and the remote sensing images as image nodes, connect the hash nodes with a codeword distance less than or equal to the preset codeword distance, connect the target nodes with the hash nodes corresponding to their discrete hash codes, and connect the image nodes with the target nodes corresponding to each target subgraph they contain to construct a hash graph.

[0100] In this embodiment, for each discrete hash code of the target sub - graph, its Hamming distance from other discrete hash codes is calculated respectively. The Hamming distance can quantify the difference degree between two hash codes. The smaller the Hamming distance, the more similar the two images are. Sort the Hamming distances from small to large, and only retain the relationships where the Hamming distance does not exceed min{MinDist, d}, where MinDist is the minimum value among the Hamming distances between this discrete hash code and all other discrete hash codes; the preset codeword distance is min{MinDist, d}; d is a preset distance, which takes the value of 2 in this embodiment.

[0101] Generally speaking, this embodiment can achieve the following beneficial effects:

[0102] (1) When training the hash model of the present invention, the target detection module is first used to identify potential targets, which can minimize the introduction of invalid information during feature extraction. After mapping the targets to a unified hash space, the model has the ability to quantitatively express similarity relationships, making up for the limitation that traditional target detection modules can only output discrete category information, and enabling content - based similarity target queries to be possible.

[0103] (2) Based on the hash codes generated by the hash model of the present invention, different modalities are projected into a unified hash space, and a semantic graph is constructed based on the similarity between the hash codes. It can gather data with similar content to adjacent positions in the graph. Through graph search, traversal - type search can be avoided, thereby improving the retrieval efficiency for content similarity.

[0104] (3) During the model training process of the present invention, by using information such as the category and confidence of target detection, the parameters of the hash model are guided to be updated in the desired direction. Without relying on manually labeled data, it can be trained on a large - scale unlabeled remote - sensing image dataset, solving the problem that it is difficult to obtain sufficient labels for large - scale data in practical applications and reducing the training cost.

[0105] In summary, in the training process of the hash code generation model of the present invention, by making full use of the information obtained from target localization, paying as much attention as possible to the image regions containing targets, reducing the introduction of invalid information, guiding the parameters of the hash code generation model, so that in the hash codes extracted by the hash module, hash codes with similar semantics can be closely mapped together, and the discriminability of the hash codes is enhanced, thereby improving the accuracy of multi - target remote - sensing image retrieval.

[0106] In the second aspect, the present invention provides a method for generating hash codes of remote - sensing images, including: inputting the remote - sensing image to be generated with hash codes into a hash code generation model to obtain the hash codes of the remote - sensing image; wherein, the hash codes of the remote - sensing image include the discrete hash codes of its target sub - graphs.

[0107] Among them, the hash code generation model is constructed by using the hash code generation model construction method provided in the first aspect of the present invention.

[0108] The related technical solutions are the same as the hash code generation model construction method provided in the first aspect of the present invention, and are not limited here.

[0109] In a third aspect, the present invention provides a hash retrieval method for remote sensing images, as Figure 2 shown, including:

[0110] S1. Input the single-target image to be retrieved into the hash code generation model to obtain the corresponding discrete hash code as the target hash code;

[0111] S2. Determine whether the target hash code exists in the hash map. If not, go to S3; otherwise, go to S4;

[0112] S3. Add the target hash code as a new hash node to the hash map and connect it to the remaining hash nodes in the hash map whose codeword distances from it are less than or equal to the preset codeword distance;

[0113] S4. In the hash map, starting from the target hash code as the starting hash node, find the neighboring hash nodes within the preset number of steps, and select the K neighboring hash nodes with the smallest codeword distance from the target hash code as the candidate hash nodes; K≥1; In an optional embodiment, K is 5.

[0114] S5. In the hash map, find the target nodes connected to the candidate hash nodes as the candidate target nodes; in the hash map, find the image nodes connected to the candidate target nodes as the candidate image nodes; count the number of candidate target nodes connected to each found candidate image node, and sort the candidate image nodes in descending order of the number of candidate target nodes connected to them, and output them as the retrieval result;

[0115] Among them, the hash code generation model and the hash map set are obtained by using the hash code generation model construction method provided in the first aspect of the present invention.

[0116] The related technical solutions are the same as the hash code generation model construction method provided in the first aspect of the present invention, and are not limited here.

[0117] In a fourth aspect, the present invention provides an electronic device, including: a memory and a processor, the memory stores a computer program, and the processor executes the method provided in the first aspect, the second aspect or the third aspect of the present invention when executing the computer program.

[0118] The related technical solutions are the same as the methods provided in the first aspect, the second aspect or the third aspect of the present invention, and are not limited here.

[0119] In a fifth aspect, the present invention further provides a computer-readable storage medium, which includes a stored computer program. When the computer program is run by a processor, it controls the device where the storage medium is located to execute the method provided in the first, second, or third aspect of the present invention.

[0120] The related technical solutions are the same as the methods provided in the first, second, or third aspect of the present invention, and are not limited here.

[0121] In a sixth aspect, the invention further provides a computer program product, including a computer program / instructions. When the computer program / instructions are executed by a processor, they implement the method provided in the first, second, or third aspect of the present invention.

[0122] The related technical solutions are the same as the methods provided in the first, second, or third aspect of the present invention, and are not limited here.

[0123] Those skilled in the art can easily understand that the above are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for constructing a hash code generation model of remote sensing images, characterized in that, Comprising: Training the hash code generation model in one or more batches to obtain a trained hash code generation model; The hash code generation model includes: a target detection module for performing target detection on a remote sensing image to obtain the position information of each target in the remote sensing image, and extracting the targets in the remote sensing image from the remote sensing image to obtain the target sub-images included in the remote sensing image; a feature extraction module for extracting the features of the target sub-images; a hash module for converting the features of the target sub-images into corresponding continuous hash codes; a mapping module for binary mapping the continuous hash codes of the target sub-images into corresponding discrete hash codes; the hash code of the remote sensing image includes the discrete hash codes of its target sub-images; Wherein, the following operations are performed in each training batch: Inputting the remote sensing images in the current training batch into the hash code generation model to obtain the continuous hash codes and discrete hash codes of each target sub-image in each remote sensing image; Constructing a training objective including minimizing the total quantization loss, and training the hash code generation model based on the training objective; wherein, the total quantization loss is the sum of the quantization losses of all target sub-images in the current training batch; the quantization loss of a target sub-image is used to measure the difference degree between the continuous hash code and the discrete hash code of the target sub-image.

2. The method for constructing a hash code generation model according to claim 1, wherein The target detection module also obtains the category information of each target in the remote sensing image when performing target detection on the remote sensing image; the category information includes: the category and confidence of the target; The target sub-images obtained by the target detection module are target sub-images with a confidence greater than or equal to a preset threshold; The hash code of the remote sensing image includes the discrete hash codes of the target sub-images with an in-built confidence greater than or equal to the preset threshold.

3. The method for constructing a hash code generation model according to claim 2, wherein The training objective also includes: minimizing the positive difference loss and maximizing the negative difference loss Among them, bs is the number of target subgraphs in the previous training batch; n(P i ) is the number of target subgraphs in the positive sample set P i of the i-th target subgraph; P i is the set composed of other target subgraphs with the same category as the i-th target subgraph among all target subgraphs in the current training batch except the i-th target subgraph; w j is the confidence corresponding to the j-th target subgraph in P i ; n(N i ) is the number of target subgraphs in the negative sample set N i of the i-th target subgraph; loss ij is the difference loss between the continuous hash code H i of the i-th target subgraph and the continuous hash code H i ' of the j-th target subgraph in P j ; n(N i ) is the set composed of target subgraphs with different categories from the i-th target subgraph among all target subgraphs in the current training batch; w k is the confidence corresponding to the k-th target subgraph in N i ; loss ik is the difference loss between the continuous hash code H i of the i-th target subgraph and the continuous hash code H i ' of the k-th target subgraph in N k .

4. The method for constructing a hash code generation model according to claim 3, wherein Minimize the positive difference loss by minimizing the contrastive loss and maximize the negative difference loss wherein, the contrastive loss is is the similarity between the continuous hash code H i and the continuous hash code H j '; is the similarity between the continuous hash code H i and the continuous hash code H k '; the greater the similarity of the hash codes, the smaller the corresponding difference loss; γ is a hyperparameter; D(H i , H j ) is the codeword distance between the continuous hash code H i and the continuous hash code H j '; D(H i , H k ) is the codeword distance between the continuous hash code H i and the continuous hash code H k '.

5. The method for constructing a hash code generation model according to claim 1, wherein The mapping module uses the sign function to binary map the continuous hash codes of the target sub-images into corresponding discrete hash codes; the total quantization loss is: Among them, bs is the number of target subgraphs in the previous training batch; L i is the length of the continuous hash code of the i-th target subgraph generated by the hash code generation model; cosh(·) is the hyperbolic cosine function; H i,l is the value of the l-th dimension in the continuous hash code of the i-th target subgraph.

6. The method for constructing a hash code generation model according to any one of claims 1-5, characterized in that, Also comprising: After completing the training of the hash code generation model, retaining the discrete hash codes of each target sub-image in all the remote sensing images used for training; Using the discrete hash codes of the target sub-images as hash nodes, the target sub-images as target nodes, and the remote sensing images as image nodes, connecting the hash nodes with a codeword distance less than or equal to a preset codeword distance, connecting the target nodes with the hash nodes corresponding to their discrete hash codes, and connecting the image nodes with the target nodes corresponding to each target sub-image included therein, to construct a hash graph.

7. A method for generating a hash code of a remote sensing image, characterized in that, Comprising: Inputting the remote sensing image to be generated with a hash code into the hash code generation model to obtain the hash code of the remote sensing image; Wherein, the hash code generation model is constructed by using the hash code generation model construction method described in any one of claims 1-6.

8. A hashing retrieval method for remote sensing images, characterized in that, Comprising: S1. Inputting the single-target image to be retrieved into the hash code generation model to obtain the corresponding discrete hash code as the target hash code; S2. Judging whether the target hash code exists in the hash graph. If not, go to S3; otherwise, go to S4; S3. Adding the target hash code as a new hash node to the hash graph and connecting it with the remaining hash nodes in the hash graph with a codeword distance less than or equal to the preset codeword distance; S4. In the hash map, using the target hash code as the starting hash node, search for neighboring hash nodes with the number of search steps within a preset range, and select K neighboring hash nodes with the smallest codeword distance from the target hash code as candidate hash nodes; K≥1; S5. Search for target nodes connected to the candidate hash nodes in the hash map as candidate target nodes; Search for image nodes connected to the candidate target nodes in the hash map as candidate image nodes; Count the number of candidate target nodes connected to each of the found candidate image nodes, and after sorting the candidate image nodes in descending order according to the number of candidate target nodes they are connected to, output them as the retrieval result; Among them, the hash code generation model is constructed by using the hash code generation model construction method described in any one of claims 1-6; the hash map set is obtained by using the hash code generation model construction method described in claim 6.

9. An electronic device, characterized in that, It includes: A memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it executes the method described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein when the computer program is run by a processor, it controls the device where the storage medium is located to execute the method described in any one of claims 1-8.

Images