Remote sensing target retrieval method, device and equipment based on multi-path similarity graph

Abstract

The application provides a remote sensing target retrieval method, device and equipment based on a multi-path similarity graph, and relates to the technical field of remote sensing. The method comprises: acquiring a fine-grained category description text corresponding to a remote sensing target to be detected, and an optical image database to be retrieved; wherein the optical image database comprises a plurality of optical remote sensing images; and inputting the fine-grained category description text and the plurality of optical remote sensing images into a pre-trained remote sensing target fine-grained retrieval model, and determining a retrieval result corresponding to the remote sensing target from the plurality of optical remote sensing images through the remote sensing target fine-grained retrieval model. The remote sensing target fine-grained retrieval model is trained based on fine-grained category description text samples corresponding to a plurality of remote sensing target samples and a plurality of optical remote sensing image samples in an optical remote sensing image data sample library. The technical solution of the application can realize fine-grained retrieval of remote sensing targets, thereby improving the accuracy of the remote sensing target retrieval result.

Description

Technical Field

[0001] This application relates to the field of remote sensing technology, and in particular to a remote sensing target retrieval method, apparatus and device based on multi-path similarity maps. Background Technology

[0002] With the deep integration and widespread application of deep learning technology in the field of remote sensing, remote sensing image retrieval technology has made groundbreaking progress. For the retrieval of conventional remote sensing targets, such as aircraft, ships, and vehicles, the main methods are single-modal (image search) and cross-modal (text search) retrieval methods based on convolutional neural networks and Transformer architectures, which achieve accurate retrieval of coarse-grained target categories.

[0003] However, unlike the conventional remote sensing target retrieval methods mentioned above, for more challenging fine-grained remote sensing target retrieval, such as different models of aircraft under the aircraft category, the core is to distinguish the subtle visual feature differences between different models of aircraft. However, the methods mentioned above cannot effectively distinguish the subtle visual feature differences between different models of aircraft, resulting in low accuracy of remote sensing target retrieval results.

[0004] Therefore, how to achieve fine-grained retrieval of remote sensing targets, thereby improving the accuracy of remote sensing target retrieval results, is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] This application provides a remote sensing target retrieval method, apparatus, and device based on multi-path similarity maps to address the shortcomings of existing technologies in achieving fine-grained retrieval of remote sensing targets, thereby improving the accuracy of remote sensing target retrieval results.

[0006] This application provides a remote sensing target retrieval method based on multi-path similarity maps, the method comprising:

[0007] The fine-grained category description text corresponding to the remote sensing target to be detected, and the optical image database to be retrieved are obtained; wherein, the optical image database includes multiple optical remote sensing images;

[0008] The fine-grained category description text and the multiple optical remote sensing images are both input into a pre-trained fine-grained remote sensing target retrieval model. The fine-grained remote sensing target retrieval model is then used to determine the retrieval results corresponding to the remote sensing target from the multiple optical remote sensing images.

[0009] The fine-grained retrieval model for remote sensing targets is trained based on fine-grained category description text samples corresponding to multiple remote sensing target samples and multiple optical remote sensing image samples in the optical remote sensing image data sample library.

[0010] According to the remote sensing target retrieval method based on multi-path similarity maps provided in this application, the step of inputting the fine-grained category description text and the multiple optical remote sensing images into a pre-trained fine-grained remote sensing target retrieval model, and determining the retrieval result corresponding to the remote sensing target from the multiple optical remote sensing images through the fine-grained remote sensing target retrieval model, includes:

[0011] The fine-grained category description text is input into the text encoder in the fine-grained retrieval model of the remote sensing target to obtain multi-channel high-level text features;

[0012] For each optical remote sensing image, the optical remote sensing image is input into the image encoder in the fine-grained retrieval model of the remote sensing target to obtain multi-channel high-level image features. The multi-channel high-level text features and the multi-channel high-level image features are then input into the similarity map calculation module in the fine-grained retrieval model of the remote sensing target to obtain a multi-channel similarity map between the fine-grained category description text and the optical remote sensing image.

[0013] In the multi-path similarity map, the target optical remote sensing image corresponding to the maximum similarity value calculated by the deepest similarity map is the retrieval result corresponding to the remote sensing target.

[0014] According to the remote sensing target retrieval method based on multi-path similarity maps provided in this application, the training process of the fine-grained remote sensing target retrieval model based on fine-grained category description text samples corresponding to multiple remote sensing target samples and multiple optical remote sensing image samples in an optical remote sensing image data sample library includes:

[0015] The fine-grained category description text samples corresponding to the remote sensing target samples are input into the text encoder in the initial fine-grained remote sensing target retrieval model to obtain multi-channel high-level text feature samples;

[0016] For each optical remote sensing image sample, the optical remote sensing image sample is input into the image encoder in the initial fine-grained remote sensing target retrieval model to obtain multi-channel high-level image feature samples; the multi-channel high-level text feature samples and the multi-channel high-level image feature samples are input into the similarity map calculation module in the initial fine-grained remote sensing target retrieval model to obtain multi-channel similarity map samples; based on the multi-channel similarity map samples, a multi-channel distillation loss function corresponding to the optical remote sensing image sample is constructed;

[0017] Based on the multi-path distillation loss function corresponding to each of the optical remote sensing image samples, the model parameters of the initial remote sensing target fine-grained retrieval model are updated, and the remote sensing target fine-grained retrieval model is trained.

[0018] According to the remote sensing target retrieval method based on multi-path similarity maps provided in this application, the step of constructing a multi-path distillation loss function corresponding to the optical remote sensing image sample based on the multi-path similarity map samples includes:

[0019] A first distillation loss function is constructed based on the multi-path similarity map samples, converting the image dimension to the text dimension.

[0020] A second distillation loss function is constructed based on the multi-path similarity map samples, converting the text dimension to the image dimension.

[0021] Based on the first distillation loss function and the second distillation loss function, the multi-channel distillation loss function corresponding to the optical remote sensing image sample is determined.

[0022] According to the remote sensing target retrieval method based on multi-path similarity maps provided in this application, the construction of a first distillation loss function from image dimension to text dimension based on the multi-path similarity map samples includes:

[0023] based on Construct the first distillation loss function from the image dimension to the text dimension;

[0024] in, This represents the first distillation loss function. This indicates the total number of elements included in the multipath similarity map sample. This represents the total number of pathways in the multipath similarity map samples. Indicates the first of the multi-path similarity map samples Pathway similarity map Indicates the first of the multi-path similarity map samples One element, Indicates the first The first path similarity graph The similarity value from the image dimension to the text dimension for each element. Indicates the first The first path similarity graph The similarity value between the image dimension and the text dimension corresponding to each element.

[0025] According to the remote sensing target retrieval method based on multi-path similarity maps provided in this application, the construction of a second distillation loss function from text dimension to image dimension based on the multi-path similarity map samples includes:

[0026] based on Construct the second distillation loss function from the text dimension to the image dimension;

[0027] in, This represents the second distillation loss function. This indicates the total number of elements included in the multipath similarity map sample. This represents the total number of pathways in the multipath similarity map samples. Indicates the first of the multi-path similarity map samples Pathway similarity map Indicates the first of the multi-path similarity map samples One element, Indicates the first The first path similarity graph The similarity value from the text dimension to the image dimension for each element. Indicates the first The first path similarity graph The similarity value between the text dimension and the image dimension of each element.

[0028] According to the remote sensing target retrieval method based on multi-path similarity maps provided in this application, the step of determining the multi-path distillation loss function corresponding to the optical remote sensing image sample based on the first distillation loss function and the second distillation loss function includes:

[0029] based on Determine the multi-channel distillation loss function corresponding to the optical remote sensing image sample;

[0030] in, This represents the multi-channel distillation loss function. This represents the first distillation loss function. This represents the second distillation loss function.

[0031] According to the remote sensing target retrieval method based on multi-path similarity maps provided in this application, the step of obtaining the fine-grained category description text corresponding to the remote sensing target to be detected includes:

[0032] Determine the fine-grained category to which the remote sensing target belongs;

[0033] Find the mapping relationship between the preset fine-grained categories and the fine-grained category description text, and determine the fine-grained category description text corresponding to the fine-grained category to which the remote sensing target belongs as the fine-grained category description text.

[0034] This application also provides a remote sensing target retrieval method, apparatus, and device based on multi-path similarity maps, comprising:

[0035] The acquisition unit is used to acquire fine-grained category description text corresponding to the remote sensing target to be detected, and an optical image database to be retrieved; wherein, the optical image database includes multiple optical remote sensing images;

[0036] The retrieval unit is used to input the fine-grained category description text and the plurality of optical remote sensing images into a pre-trained fine-grained retrieval model for remote sensing targets, and to determine the retrieval result corresponding to the remote sensing target from the plurality of optical remote sensing images through the fine-grained retrieval model for remote sensing targets;

[0037] The fine-grained retrieval model for remote sensing targets is trained based on fine-grained category description text samples corresponding to multiple remote sensing target samples and multiple optical remote sensing image samples in the optical remote sensing image data sample library.

[0038] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the remote sensing target retrieval method based on multi-path similarity maps as described in any of the preceding claims.

[0039] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the remote sensing target retrieval method based on a multi-path similarity map as described in any of the preceding claims.

[0040] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the remote sensing target retrieval method based on a multi-path similarity map as described in any of the preceding claims.

[0041] The remote sensing target retrieval method, apparatus, and device based on multi-path similarity maps provided in this application, when performing remote sensing target retrieval, acquires fine-grained category description text corresponding to the remote sensing target to be detected, and an optical image database to be retrieved; wherein, the optical image database includes multiple optical remote sensing images; and inputs both the fine-grained category description text and the multiple optical remote sensing images into a pre-trained fine-grained remote sensing target retrieval model, and determines the retrieval result corresponding to the remote sensing target from the multiple optical remote sensing images through the fine-grained remote sensing target retrieval model. The fine-grained remote sensing target retrieval model is trained based on fine-grained category description text samples corresponding to multiple remote sensing target samples and multiple optical remote sensing image samples from the optical remote sensing image data sample database. In this way, by acquiring the fine-grained category description text corresponding to the remote sensing target to be detected and combining it with the pre-trained fine-grained remote sensing target retrieval model, fine-grained retrieval of remote sensing targets can be achieved, thereby effectively improving the accuracy of remote sensing target retrieval results. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart illustrating a remote sensing target retrieval method based on a multi-path similarity map, provided in an embodiment of this application.

[0044] Figure 2 This is a schematic diagram illustrating a process for determining the retrieval results corresponding to a remote sensing target from multiple optical remote sensing images using a fine-grained remote sensing target retrieval model, as provided in an embodiment of this application.

[0045] Figure 3 This is a flowchart illustrating a training method for a fine-grained remote sensing target retrieval model, as provided in an embodiment of this application.

[0046] Figure 4 This is a schematic diagram of the structure of a remote sensing target retrieval device based on a multi-path similarity map, provided in an embodiment of this application.

[0047] Figure 5 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0049] In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone, where A and B can be singular or plural. In the textual description of this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0050] The technical solution provided in this application can be applied to remote sensing target retrieval scenarios, such as the detection of different aircraft models under different aircraft categories. Taking the detection of different aircraft models under different aircraft categories as an example, considering fine-grained remote sensing target retrieval, its core lies in distinguishing the subtle visual feature differences between different aircraft models. However, existing conventional remote sensing target retrieval methods cannot effectively distinguish the subtle visual feature differences between different aircraft models, resulting in low accuracy of remote sensing target retrieval results.

[0051] To achieve fine-grained retrieval of remote sensing targets and thus improve the accuracy of retrieval results, a method can be employed during the retrieval process. This method involves acquiring fine-grained category description text corresponding to the remote sensing target to be detected, along with a database of optical images to be retrieved. The optical image database includes multiple optical remote sensing images. Both the fine-grained category description text and the multiple optical remote sensing images are input into a pre-trained fine-grained retrieval model for remote sensing targets. The model then determines the retrieval result corresponding to the remote sensing target from the multiple optical remote sensing images. By acquiring the fine-grained category description text corresponding to the remote sensing target to be detected and combining it with the pre-trained fine-grained retrieval model, fine-grained retrieval of remote sensing targets can be achieved, effectively improving the accuracy of retrieval results.

[0052] It is understood that the execution entity of the remote sensing target retrieval method based on multi-path similarity maps provided in this application can be an electronic device such as a computer, server, or specially set remote sensing target retrieval equipment, or it can be a remote sensing target retrieval device based on multi-path similarity maps set in such electronic device. The remote sensing target retrieval device based on multi-path similarity maps can be implemented by software, hardware, or a combination of both, and can be set according to actual needs.

[0053] The remote sensing target retrieval method based on multi-path similarity maps provided in this application will be described in detail below through several specific embodiments. It is understood that these specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0054] Figure 1 This application provides a flowchart illustrating a remote sensing target retrieval method based on a multi-path similarity map. For example, please refer to [link to relevant documentation]. Figure 1 As shown, this remote sensing target retrieval method based on multi-path similarity maps may include:

[0055] S101. Obtain the fine-grained category description text corresponding to the remote sensing target to be detected, and the optical image database to be retrieved; wherein, the optical image database includes multiple optical remote sensing images.

[0056] Among them, the fine-grained category description text corresponding to the remote sensing target refers to the category description text of the fine-grained category to which the remote sensing target belongs. It can be expressed as "an optical remote sensing image of [fine-grained category]", or as "an optical remote sensing image containing the [parent] target of [fine-grained category]". Different fine-grained categories have their own category description text.

[0057] Understandably, fine-grained category description text refers to a text format that uses natural language to describe in detail the specific category, type, or characteristics of remote sensing targets. This type of description text not only includes basic category information of the remote sensing target but also further refines it to details such as subcategories, types, and functions, achieving more accurate target identification and interpretation, thus providing a basis for subsequent remote sensing target detection.

[0058] For example, if the coarse-grained category [parent category] includes aircraft and ships, then the different subcategories under this coarse-grained category, i.e., the [fine-grained categories], can include ARJ21, A220, A320, A330, Boeing 737, Boeing 747, Boeing 777, Boeing 787, cargo ships, fishing boats, tugboats, and military ships. In this scenario, the remote sensing target can be a target of one of the above 12 different subcategories of fine-grained category, so as to achieve cross-modal retrieval that is refined to different subcategories under the coarse-grained category, which can greatly improve the ability to extract text features and image features of fine-grained categories.

[0059] For example, when obtaining the fine-grained category description text corresponding to the remote sensing target to be detected, the fine-grained category to which the remote sensing target belongs can be determined first; and the mapping relationship between the preset fine-grained category and the fine-grained category description text can be found, and the fine-grained category description text corresponding to the fine-grained category to which the remote sensing target belongs can be determined as the fine-grained category description text.

[0060] When performing remote sensing target retrieval, in addition to obtaining the fine-grained category description text corresponding to the remote sensing target to be detected, it is also necessary to obtain the optical image database to be detected. This optical image database can be understood as a collection of optical remote sensing images aggregating different fine-grained categories. Taking a coarse-grained category [parent category] including aircraft and ships as an example, the optical image database could be a collection of optical remote sensing images corresponding to the aforementioned 12 different subcategories.

[0061] S102. Input the fine-grained category description text and multiple optical remote sensing images into the pre-trained fine-grained remote sensing target retrieval model, and use the fine-grained remote sensing target retrieval model to determine the retrieval results corresponding to the remote sensing target from multiple optical remote sensing images.

[0062] The fine-grained retrieval model for remote sensing targets is trained based on fine-grained category description text samples corresponding to multiple remote sensing target samples and multiple optical remote sensing image samples in the optical remote sensing image data sample library.

[0063] As can be seen from the embodiments of this application, when performing remote sensing target retrieval, fine-grained category description text corresponding to the remote sensing target to be detected and an optical image database to be retrieved can be obtained; wherein, the optical image database includes multiple optical remote sensing images; and the fine-grained category description text and multiple optical remote sensing images are both input into a pre-trained fine-grained remote sensing target retrieval model, and the retrieval result corresponding to the remote sensing target is determined from multiple optical remote sensing images by the fine-grained remote sensing target retrieval model. In this way, by obtaining the fine-grained category description text corresponding to the remote sensing target to be detected and combining it with the pre-trained fine-grained remote sensing target retrieval model, fine-grained retrieval of remote sensing targets can be achieved, thereby effectively improving the accuracy of remote sensing target retrieval results.

[0064] Based on the above Figure 1 In the illustrated embodiment, for example, in S102 above, both the fine-grained category description text and multiple optical remote sensing images are input into the fine-grained remote sensing target retrieval model. The specific implementation of determining the retrieval result corresponding to the remote sensing target from multiple optical remote sensing images using the fine-grained remote sensing target retrieval model can be found below. Figure 2 The example shown.

[0065] Figure 2 This application provides a flowchart illustrating a process for determining the retrieval result corresponding to a remote sensing target from multiple optical remote sensing images using a fine-grained remote sensing target retrieval model. For example, see [link to relevant documentation]. Figure 2 As shown, the method may include:

[0066] S201. Input the fine-grained category description text into the text encoder in the fine-grained retrieval model of remote sensing targets to obtain multi-channel high-level text features.

[0067] For example, in this embodiment of the application, a Transformer structure can be selected as the feature encoder structure for the input text. Specifically, the CLIP (Contrastive Language-Image Pre-training) method can be used to pre-train using publicly available natural image data and text descriptions, and the ViT (Vision Transformer) network can be used as the text encoder to obtain more accurate text feature extraction capabilities. The specific settings can be configured according to actual needs.

[0068] The fine-grained category description text is input into the text encoder in the fine-grained retrieval model of remote sensing targets. For example, the text encoder can convert the fine-grained category description text into an acceptable format and capture semantic information at different levels through shallow, medium and deep pathways to form the final multi-path high-level text features.

[0069] Taking the ViT network as a text encoder as an example, when extracting multi-channel high-level text features through the text encoder in the fine-grained remote sensing target retrieval model, the following formula 1 can be used as an example:

[0070] Formula 1

[0071] in, Let X represent the text feature of the i-th pathway, X represent the fine-grained category description text of the input, and TextEncoder represent the text encoder of the ViT network.

[0072] S202. For each optical remote sensing image, the optical remote sensing image is input into the image encoder in the fine-grained retrieval model of remote sensing target to obtain multi-channel high-level image features. The multi-channel high-level text features and multi-channel high-level image features are then input into the similarity map calculation module in the fine-grained retrieval model of remote sensing target to obtain a multi-channel similarity map between the fine-grained category description text and the optical remote sensing image.

[0073] Among them, in the multi-path similarity map, the target optical remote sensing image corresponding to the maximum similarity value calculated by the deepest similarity map is the retrieval result corresponding to the remote sensing target.

[0074] The multi-path similarity graph can be understood as a similarity matrix, where each element represents the similarity score between a pair of text features and image features, which is used for subsequent matching, retrieval, or alignment tasks.

[0075] For example, in this embodiment of the application, a Transformer structure can also be selected as the feature encoder structure for the input text. Specifically, the CLIP method can be used to pre-train using publicly available natural image data and text descriptions, and the ViT (Vision Transformer) network can be used as the image encoder to obtain more accurate image feature extraction capabilities. The specific settings can be configured according to actual needs.

[0076] The optical remote sensing image is input into the image encoder in the fine-grained retrieval model of the remote sensing target. For example, the optical remote sensing image can be converted into an acceptable format by the image encoder, and semantic information at different levels can be captured through shallow, medium and deep pathways to form the final multi-channel high-level image features.

[0077] Taking the ViT network as an image encoder as an example, it can perform position encoding processing on optical remote sensing images, and after fusing with the Gram matrix, input it into the image encoder in the fine-grained retrieval model of remote sensing targets to extract multi-channel high-level image features, as shown in Formula 2 below:

[0078] Formula 2

[0079] in, Let I represent the intermediate feature of the i-th path, I represent the input optical remote sensing image, and ImgEncoder represent the image encoder.

[0080] For example, the Gram matrix mentioned above can be a Gram matrix obtained by inputting an optical remote sensing image into a frozen lightweight convolutional neural network (CNN) to extract preliminary image features and learn the features. The specific settings can be configured according to actual needs.

[0081] After obtaining the multi-path high-level text features and multi-path high-level image features, the similarity map calculation module can be used to calculate the multi-path similarity map between the fine-grained category description text and the optical remote sensing image. For example, the multi-path high-level text features and multi-path high-level image features can first be normalized using the similarity map calculation module, for example, by mapping them to the same dimensional space through fully connected layers, linear transformations, or dimensionality reduction methods. Secondly, for each pair of high-level text features and high-level image features, their similarity matrix can be calculated, for example, using a cosine similarity matrix, a dot product similarity matrix, or a negative Euclidean distance as the similarity matrix. Then, the similarity matrices of all paths are fused to obtain the multi-path similarity map between the fine-grained category description text and the optical remote sensing image. Similarly, the multi-path similarity map between the fine-grained category description text and each optical remote sensing image in the optical image database can be calculated. Then, the optical remote sensing image corresponding to the maximum similarity value calculated from the deepest similarity map in all multi-path similarity maps is determined as the retrieval result corresponding to the remote sensing target.

[0082] By combining fine-grained category description text corresponding to remote sensing targets to extract multi-path high-level text features and combining them with multi-path high-level image features to determine multi-path similarity maps, fine-grained features can be distilled from optical remote sensing images to solve the problem of intra-class discrimination difficulty caused by high semantic similarity. Combined with a pre-trained fine-grained retrieval model for remote sensing targets, fine-grained retrieval of remote sensing targets can be achieved, thereby effectively improving the accuracy of remote sensing target retrieval results.

[0083] Based on any of the above embodiments, for example, the training process of the above-mentioned fine-grained remote sensing target retrieval model based on fine-grained category description text samples corresponding to multiple remote sensing target samples and multiple optical remote sensing image samples in the optical remote sensing image data sample library can be referred to below. Figure 3 The example shown.

[0084] Figure 3 This application provides a flowchart illustrating the training of a fine-grained remote sensing target retrieval model. For example, see [link to relevant documentation]. Figure 3 As shown, the method may include:

[0085] S301. Input the fine-grained category description text sample corresponding to the remote sensing target sample into the text encoder in the initial remote sensing target fine-grained retrieval model to obtain multi-channel high-level text feature samples.

[0086] For example, in the embodiments of this application, the initial fine-grained retrieval model for remote sensing targets can be a Transformer network model, or of course, a CNN model, etc., which can be set according to actual needs.

[0087] It should be noted that, in this embodiment of the application, the specific implementation of extracting multiple high-level text feature samples through the text encoder in the initial remote sensing target fine-grained retrieval model is similar to the specific implementation of extracting multiple high-level text features through the text encoder in the remote sensing target fine-grained retrieval model in S201 above. Please refer to the relevant description in S201 above. Here, this embodiment of the application will not repeat the description.

[0088] S302. For each optical remote sensing image sample, input the optical remote sensing image sample into the image encoder in the initial fine-grained retrieval model of remote sensing target to obtain multi-channel high-level image feature samples; input the multi-channel high-level text feature samples and the multi-channel high-level image feature samples into the similarity map calculation module in the initial fine-grained retrieval model of remote sensing target to obtain multi-channel similarity map samples; construct the multi-channel distillation loss function corresponding to the optical remote sensing image sample based on the multi-channel similarity map samples.

[0089] It should be noted that, in the embodiments of this application, the specific implementation of extracting multiple high-level image feature samples through the image encoder in the initial fine-grained remote sensing target retrieval model, and the specific implementation of extracting multiple similarity map samples through the similarity map calculation module in the initial fine-grained remote sensing target retrieval model, are similar to the specific implementations in S202 above, namely, extracting multiple high-level image features through the image encoder in the fine-grained remote sensing target retrieval model, and extracting multiple similarity maps through the similarity map calculation module in the fine-grained remote sensing target retrieval model. Please refer to the relevant descriptions in S202 above; therefore, the embodiments of this application will not repeat them here.

[0090] For example, in the embodiments of this application, when constructing the multi-path distillation loss function corresponding to the optical remote sensing image sample based on the multi-path similarity map sample, a first distillation loss function from the image dimension to the text dimension can be constructed based on the multi-path similarity map sample; and a second distillation loss function from the text dimension to the image dimension can be constructed based on the multi-path similarity map sample; then, based on the first distillation loss function and the second distillation loss function, the multi-path distillation loss function corresponding to the optical remote sensing image sample is determined. In this way, the model parameters of the initial fine-grained remote sensing target retrieval model can be updated based on the multi-path distillation loss function corresponding to each optical remote sensing image sample, thereby training the fine-grained remote sensing target retrieval model.

[0091] For example, suppose the first The high-level text feature samples of each pathway can be denoted as: , No. The high-level image feature samples of each pathway can be denoted as: Then calculate the first The high-level text feature samples of the first pathway and the first When creating a multipath similarity map between high-level image feature samples of the first path, you can first... High-level text feature samples of each pathway and the High-level image feature samples of each pathway After normalization, we get and ; and based on the normalized first High-level text feature samples of each pathway , and based on the normalized first High-level image feature samples of each pathway Calculate the first Image-to-text similarity graph for each pathway , Conversely, calculate the first... Similarity graph from text dimension to image dimension for each pathway It can be directly characterized as .

[0092] Based on the above description, for example, in an embodiment of this application, constructing a first distillation loss function from the image dimension to the text dimension based on multi-path similarity map samples may include:

[0093] based on Construct the first distillation loss function from the image dimension to the text dimension.

[0094] in, This represents the first distillation loss function. This indicates the total number of elements included in the multipath similarity graph sample. This represents the total number of pathways in the multipath similarity map samples. Indicates the first sample in the multi-path similarity graph Pathway similarity map Indicates the first sample in the multi-path similarity graph One element, Indicates the first The first path similarity graph The similarity value from the image dimension to the text dimension for each element. Indicates the first The first path similarity graph The similarity value between the image dimension and the text dimension corresponding to each element.

[0095] For example, in an embodiment of this application, constructing a second distillation loss function from text dimension to image dimension based on multi-path similarity map samples includes:

[0096] based on Construct a second distillation loss function from the text dimension to the image dimension;

[0097] in, This represents the second distillation loss function. This indicates the total number of elements included in the multi-path similarity graph. This represents the total number of channels in the multipath similarity graph. Represents the first path in the multi-path similarity graph. Pathway similarity map Indicates the first sample in the multi-path similarity graph One element, Indicates the first The first path similarity graph The similarity value from the text dimension to the image dimension for each element. Indicates the first The first path similarity graph The similarity value between the text dimension and the image dimension of each element.

[0098] After determining the first distillation loss function and the second distillation loss function respectively, for example, in an embodiment of this application, determining the multi-channel distillation loss function corresponding to the remote sensing target sample based on the first distillation loss function and the second distillation loss function may include:

[0099] based on Determine the multi-channel distillation loss function corresponding to the remote sensing target sample;

[0100] in, This represents the loss function of multi-channel distillation. This represents the first distillation loss function. This represents the second distillation loss function.

[0101] Based on the above description, for each optical remote sensing image sample, a multi-path distillation loss function is constructed based on the multi-path similarity map. This can enhance the distillation process from the text dimension to the image dimension, as well as the distillation process from the image dimension to the text dimension. Furthermore, it can also strengthen the distillation transfer from the deep similarity map to the shallow similarity map.

[0102] After obtaining the multi-channel distillation loss function corresponding to each optical remote sensing image sample, the following step S303 can be executed:

[0103] S303. Based on the multi-channel distillation loss function corresponding to each optical remote sensing image sample, update the model parameters of the initial remote sensing target fine-grained retrieval model and train to obtain the remote sensing target fine-grained retrieval model.

[0104] When updating the model parameters of the initial fine-grained remote sensing target retrieval model based on the multi-channel distillation loss function corresponding to each optical remote sensing image sample, the average distillation loss function corresponding to the multi-channel distillation loss function of each optical remote sensing image sample can be calculated, and the model parameters of the initial fine-grained remote sensing target retrieval model can be updated based on the average distillation loss function until the model update conditions are met, such as the average distillation loss function being less than a preset threshold, or the number of updates being greater than or equal to a preset number, thereby training the fine-grained remote sensing target retrieval model.

[0105] As can be seen, in this embodiment of the application, when training the fine-grained retrieval model for remote sensing targets, the fine-grained category description text samples corresponding to the remote sensing target samples can be input into the text encoder in the initial fine-grained retrieval model for remote sensing targets to obtain multi-path high-level text feature samples; for each optical remote sensing image sample, the optical remote sensing image sample is input into the image encoder in the initial fine-grained retrieval model for remote sensing targets to obtain multi-path high-level image feature samples; and the multi-path high-level text feature samples and the multi-path high-level image feature samples are input into the similarity map calculation module in the initial fine-grained retrieval model for remote sensing targets to obtain multi-path similarity map samples; a multi-path distillation loss function corresponding to the optical remote sensing image samples is constructed based on the multi-path similarity map samples; and the model parameters of the initial fine-grained retrieval model for remote sensing targets are updated based on the multi-path distillation loss function corresponding to each optical remote sensing image sample, thereby training the fine-grained retrieval model for remote sensing targets. This method involves calculating multi-path similarity maps using multiple high-level semantic feature samples extracted from fine-grained category description text samples and multiple high-level image feature samples extracted from optical remote sensing image samples. Then, based on the distillation loss function from the text dimension to the image dimension and the distillation loss function from the image dimension to the text dimension, the initial fine-grained remote sensing target retrieval model is trained. The resulting fine-grained remote sensing target retrieval model can effectively achieve fine-grained retrieval of remote sensing targets, thereby significantly improving the accuracy of remote sensing target retrieval results.

[0106] The remote sensing target retrieval device based on multi-path similarity maps provided in this application is described below. The remote sensing target retrieval device based on multi-path similarity maps described below can be referred to in correspondence with the remote sensing target retrieval method based on multi-path similarity maps described above.

[0107] Figure 4 A schematic diagram of a remote sensing target retrieval device based on a multi-path similarity map provided in this application embodiment is shown below. For example, please refer to... Figure 4 As shown, the remote sensing target retrieval device 40 based on multi-path similarity maps may include:

[0108] The acquisition unit 401 is used to acquire fine-grained category description text corresponding to the remote sensing target to be detected, and an optical image database to be retrieved; wherein, the optical image database includes multiple optical remote sensing images;

[0109] The retrieval unit 402 is used to input the fine-grained category description text and the plurality of optical remote sensing images into a pre-trained fine-grained retrieval model for remote sensing targets, and to determine the retrieval result corresponding to the remote sensing target from the plurality of optical remote sensing images through the fine-grained retrieval model for remote sensing targets;

[0110] The fine-grained retrieval model for remote sensing targets is trained based on fine-grained category description text samples corresponding to multiple remote sensing target samples and multiple optical remote sensing image samples in the optical remote sensing image data sample library.

[0111] For example, in an embodiment of this application, the retrieval unit 402 is used to input both the fine-grained category description text and the plurality of optical remote sensing images into a pre-trained fine-grained remote sensing target retrieval model, and determine the retrieval result corresponding to the remote sensing target from the plurality of optical remote sensing images through the fine-grained remote sensing target retrieval model, including:

[0112] The fine-grained category description text is input into the text encoder in the fine-grained retrieval model of the remote sensing target to obtain multi-channel high-level text features;

[0113] For each optical remote sensing image, the optical remote sensing image is input into the image encoder in the fine-grained retrieval model of the remote sensing target to obtain multi-channel high-level image features. The multi-channel high-level text features and the multi-channel high-level image features are then input into the similarity map calculation module in the fine-grained retrieval model of the remote sensing target to obtain a multi-channel similarity map between the fine-grained category description text and the optical remote sensing image.

[0114] In the multi-path similarity map, the target optical remote sensing image corresponding to the maximum similarity value calculated by the deepest similarity map is the retrieval result corresponding to the remote sensing target.

[0115] For example, in this embodiment of the application, the training process of the fine-grained remote sensing target retrieval model based on fine-grained category description text samples corresponding to multiple remote sensing target samples and multiple optical remote sensing image samples in the optical remote sensing image data sample library includes:

[0116] The fine-grained category description text samples corresponding to the remote sensing target samples are input into the text encoder in the initial fine-grained remote sensing target retrieval model to obtain multi-channel high-level text feature samples;

[0117] For each optical remote sensing image sample, the optical remote sensing image sample is input into the image encoder in the initial fine-grained remote sensing target retrieval model to obtain multi-channel high-level image feature samples; the multi-channel high-level text feature samples and the multi-channel high-level image feature samples are input into the similarity map calculation module in the initial fine-grained remote sensing target retrieval model to obtain multi-channel similarity map samples; based on the multi-channel similarity map samples, a multi-channel distillation loss function corresponding to the optical remote sensing image sample is constructed;

[0118] Based on the multi-path distillation loss function corresponding to each of the optical remote sensing image samples, the model parameters of the initial remote sensing target fine-grained retrieval model are updated, and the remote sensing target fine-grained retrieval model is trained.

[0119] For example, in an embodiment of this application, constructing the multipath distillation loss function corresponding to the optical remote sensing image sample based on the multipath similarity map sample includes:

[0120] A first distillation loss function is constructed based on the multi-path similarity map samples, converting the image dimension to the text dimension.

[0121] A second distillation loss function is constructed based on the multi-path similarity map samples, converting the text dimension to the image dimension.

[0122] Based on the first distillation loss function and the second distillation loss function, the multi-channel distillation loss function corresponding to the optical remote sensing image sample is determined.

[0123] For example, in an embodiment of this application, constructing a first distillation loss function from image dimension to text dimension based on the multi-path similarity map samples includes:

[0124] based on Construct the first distillation loss function from the image dimension to the text dimension;

[0125] in, This represents the first distillation loss function. This indicates the total number of elements included in the multipath similarity map sample. This represents the total number of pathways in the multipath similarity map samples. Indicates the first of the multi-path similarity map samples Pathway similarity map Indicates the first of the multi-path similarity map samples One element, Indicates the first The first path similarity graph The similarity value from the image dimension to the text dimension for each element. Indicates the first The first path similarity graph The similarity value between the image dimension and the text dimension corresponding to each element.

[0126] For example, in an embodiment of this application, constructing a second distillation loss function from text dimension to image dimension based on the multi-path similarity map samples includes:

[0127] based on Construct the second distillation loss function from the text dimension to the image dimension;

[0128] in, This represents the second distillation loss function. This indicates the total number of elements included in the multipath similarity map sample. This represents the total number of pathways in the multipath similarity map samples. Indicates the first of the multi-path similarity map samples Pathway similarity map Indicates the first of the multi-path similarity map samples One element, Indicates the first The first path similarity graph The similarity value from the text dimension to the image dimension for each element. Indicates the first The first path similarity graph The similarity value between the text dimension and the image dimension of each element.

[0129] For example, in an embodiment of this application, determining the multi-channel distillation loss function corresponding to the optical remote sensing image sample based on the first distillation loss function and the second distillation loss function includes:

[0130] based on Determine the multi-channel distillation loss function corresponding to the optical remote sensing image sample;

[0131] in, This represents the multi-channel distillation loss function. This represents the first distillation loss function. This represents the second distillation loss function.

[0132] For example, in an embodiment of this application, the acquisition unit 401 is used to acquire fine-grained category description text corresponding to the remote sensing target to be detected, including:

[0133] Determine the fine-grained category to which the remote sensing target belongs;

[0134] Find the mapping relationship between the preset fine-grained categories and the fine-grained category description text, and determine the fine-grained category description text corresponding to the fine-grained category to which the remote sensing target belongs as the fine-grained category description text.

[0135] The remote sensing target retrieval device 40 based on multi-path similarity maps provided in this application embodiment can execute the technical solution of the remote sensing target retrieval method based on multi-path similarity maps in any of the above embodiments. Its implementation principle and beneficial effects are similar to those of the remote sensing target retrieval method based on multi-path similarity maps. Please refer to the implementation principle and beneficial effects of the remote sensing target retrieval method based on multi-path similarity maps. It will not be repeated here.

[0136] Figure 5 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of this application, such as... Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a remote sensing target retrieval method based on a multi-path similarity map. The method includes: acquiring fine-grained category description text corresponding to the remote sensing target to be detected, and an optical image database to be retrieved; wherein the optical image database includes multiple optical remote sensing images; inputting the fine-grained category description text and the multiple optical remote sensing images into a pre-trained fine-grained remote sensing target retrieval model, and determining the retrieval result corresponding to the remote sensing target from the multiple optical remote sensing images through the fine-grained remote sensing target retrieval model; wherein the fine-grained remote sensing target retrieval model is trained based on fine-grained category description text samples corresponding to multiple remote sensing target samples and multiple optical remote sensing image samples in the optical remote sensing image data sample database.

[0137] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0138] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the remote sensing target retrieval method based on a multi-path similarity map provided by the above methods. The method includes: acquiring fine-grained category description text corresponding to the remote sensing target to be detected, and an optical image database to be retrieved; wherein the optical image database includes multiple optical remote sensing images; inputting the fine-grained category description text and the multiple optical remote sensing images into a pre-trained fine-grained remote sensing target retrieval model, and determining the retrieval result corresponding to the remote sensing target from the multiple optical remote sensing images through the fine-grained remote sensing target retrieval model; wherein the fine-grained remote sensing target retrieval model is trained based on fine-grained category description text samples corresponding to multiple remote sensing target samples and multiple optical remote sensing image samples in the optical remote sensing image data sample library.

[0139] In another aspect, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the remote sensing target retrieval method based on a multi-path similarity map provided by the methods described above. This method includes: acquiring fine-grained category description text corresponding to a remote sensing target to be detected, and an optical image database to be retrieved; wherein the optical image database includes multiple optical remote sensing images; inputting both the fine-grained category description text and the multiple optical remote sensing images into a pre-trained fine-grained remote sensing target retrieval model, and determining the retrieval result corresponding to the remote sensing target from the multiple optical remote sensing images through the fine-grained remote sensing target retrieval model; wherein the fine-grained remote sensing target retrieval model is trained based on fine-grained category description text samples corresponding to multiple remote sensing target samples and multiple optical remote sensing image samples in an optical remote sensing image data sample library.

[0140] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0141] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A remote sensing target retrieval method based on multi-path similarity maps, characterized in that, include: The fine-grained category description text corresponding to the remote sensing target to be detected, and the optical image database to be retrieved are obtained; wherein, the optical image database includes multiple optical remote sensing images; The fine-grained category description text and the multiple optical remote sensing images are both input into a pre-trained fine-grained remote sensing target retrieval model. The fine-grained remote sensing target retrieval model is then used to determine the retrieval results corresponding to the remote sensing target from the multiple optical remote sensing images. The fine-grained retrieval model for remote sensing targets is trained based on fine-grained category description text samples corresponding to multiple remote sensing target samples and multiple optical remote sensing image samples in the optical remote sensing image data sample library. The training process of the fine-grained remote sensing target retrieval model, based on fine-grained category description text samples corresponding to multiple remote sensing target samples and multiple optical remote sensing image samples in the optical remote sensing image data sample library, includes: The fine-grained category description text samples corresponding to the remote sensing target samples are input into the text encoder in the initial fine-grained remote sensing target retrieval model to obtain multi-channel high-level text feature samples; For each optical remote sensing image sample, the optical remote sensing image sample is input into the image encoder in the initial fine-grained remote sensing target retrieval model to obtain multi-channel high-level image feature samples; the multi-channel high-level text feature samples and the multi-channel high-level image feature samples are input into the similarity map calculation module in the initial fine-grained remote sensing target retrieval model to obtain multi-channel similarity map samples; based on the multi-channel similarity map samples, a multi-channel distillation loss function corresponding to the optical remote sensing image sample is constructed; Based on the multi-path distillation loss function corresponding to each of the optical remote sensing image samples, the model parameters of the initial remote sensing target fine-grained retrieval model are updated, and the remote sensing target fine-grained retrieval model is trained.

2. The method according to claim 1, characterized in that, The step of inputting the fine-grained category description text and the multiple optical remote sensing images into a pre-trained fine-grained remote sensing target retrieval model, and determining the retrieval result corresponding to the remote sensing target from the multiple optical remote sensing images through the fine-grained remote sensing target retrieval model, includes: The fine-grained category description text is input into the text encoder in the fine-grained retrieval model of the remote sensing target to obtain multi-channel high-level text features; For each optical remote sensing image, the optical remote sensing image is input into the image encoder in the fine-grained retrieval model of the remote sensing target to obtain multi-channel high-level image features. The multi-channel high-level text features and the multi-channel high-level image features are then input into the similarity map calculation module in the fine-grained retrieval model of the remote sensing target to obtain a multi-channel similarity map between the fine-grained category description text and the optical remote sensing image. In the multi-path similarity map, the target optical remote sensing image corresponding to the maximum similarity value calculated by the deepest similarity map is the retrieval result corresponding to the remote sensing target.

3. The method according to claim 1, characterized in that, The construction of the multi-path distillation loss function corresponding to the optical remote sensing image sample based on the multi-path similarity map sample includes: A first distillation loss function is constructed based on the multi-path similarity map samples, converting the image dimension to the text dimension. A second distillation loss function is constructed based on the multi-path similarity map samples, converting the text dimension to the image dimension. Based on the first distillation loss function and the second distillation loss function, the multi-channel distillation loss function corresponding to the optical remote sensing image sample is determined.

4. The method according to claim 3, characterized in that, The first distillation loss function, constructed based on the multi-path similarity map samples to convert the image dimension to the text dimension, includes: based on Construct the first distillation loss function from the image dimension to the text dimension; in, This represents the first distillation loss function. This indicates the total number of elements included in the multipath similarity map sample. This represents the total number of pathways in the multipath similarity map samples. Indicates the first of the multi-path similarity map samples Pathway similarity map Indicates the first of the multi-path similarity map samples One element, Indicates the first The first path similarity graph The similarity value from the image dimension to the text dimension for each element. Indicates the first The first path similarity graph The similarity value between the image dimension and the text dimension corresponding to each element.

5. The method according to claim 3, characterized in that, The construction of the second distillation loss function from text dimension to image dimension based on the multi-path similarity map samples includes: based on Construct the second distillation loss function from the text dimension to the image dimension; in, This represents the second distillation loss function. This indicates the total number of elements included in the multipath similarity map sample. This represents the total number of pathways in the multipath similarity map samples. Indicates the first of the multi-path similarity map samples Pathway similarity map Indicates the first of the multi-path similarity map samples One element, Indicates the first The first path similarity graph The similarity value from the text dimension to the image dimension for each element. Indicates the first The first path similarity graph The similarity value between the text dimension and the image dimension of each element.

6. The method according to claim 3, characterized in that, The step of determining the multi-path distillation loss function corresponding to the optical remote sensing image sample based on the first distillation loss function and the second distillation loss function includes: based on Determine the multi-channel distillation loss function corresponding to the optical remote sensing image sample; in, This represents the multi-channel distillation loss function. This represents the first distillation loss function. This represents the second distillation loss function.

7. The method according to claim 1 or 2, characterized in that, The acquisition of fine-grained category description text corresponding to the remote sensing target to be detected includes: Determine the fine-grained category to which the remote sensing target belongs; Find the mapping relationship between the preset fine-grained categories and the fine-grained category description text, and determine the fine-grained category description text corresponding to the fine-grained category to which the remote sensing target belongs as the fine-grained category description text.

8. A remote sensing target retrieval device based on a multi-path similarity map, characterized in that, include: The acquisition unit is used to acquire fine-grained category description text corresponding to the remote sensing target to be detected, and an optical image database to be retrieved; wherein, the optical image database includes multiple optical remote sensing images; The retrieval unit is used to input the fine-grained category description text and the plurality of optical remote sensing images into a pre-trained fine-grained retrieval model for remote sensing targets, and to determine the retrieval result corresponding to the remote sensing target from the plurality of optical remote sensing images through the fine-grained retrieval model for remote sensing targets; The fine-grained retrieval model for remote sensing targets is trained based on fine-grained category description text samples corresponding to multiple remote sensing target samples and multiple optical remote sensing image samples in the optical remote sensing image data sample library. The training process of the fine-grained remote sensing target retrieval model, based on fine-grained category description text samples corresponding to multiple remote sensing target samples and multiple optical remote sensing image samples in the optical remote sensing image data sample library, includes: The fine-grained category description text samples corresponding to the remote sensing target samples are input into the text encoder in the initial fine-grained remote sensing target retrieval model to obtain multi-channel high-level text feature samples; For each optical remote sensing image sample, the optical remote sensing image sample is input into the image encoder in the initial fine-grained remote sensing target retrieval model to obtain multi-channel high-level image feature samples; the multi-channel high-level text feature samples and the multi-channel high-level image feature samples are input into the similarity map calculation module in the initial fine-grained remote sensing target retrieval model to obtain multi-channel similarity map samples; based on the multi-channel similarity map samples, a multi-channel distillation loss function corresponding to the optical remote sensing image sample is constructed; Based on the multi-path distillation loss function corresponding to each of the optical remote sensing image samples, the model parameters of the initial remote sensing target fine-grained retrieval model are updated, and the remote sensing target fine-grained retrieval model is trained.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the remote sensing target retrieval method based on multi-path similarity maps as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the remote sensing target retrieval method based on multi-path similarity maps as described in any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the remote sensing target retrieval method based on multi-path similarity maps as described in any one of claims 1 to 7.

Images