Aerial image similarity detection method and device, electronic equipment and storage medium

Abstract

The application provides an aerial image similarity detection method and device, electronic equipment and storage medium, comprising: obtaining an aerial image set; wherein the aerial image set comprises at least two aerial images to be detected; extracting a first image feature map of each aerial image to be detected through a pre-trained image feature extraction model; wherein the image feature extraction model is trained through unlabeled aerial images, and in the training stage, the input of the image feature extraction model is an unmasked image block in the unlabeled aerial images, and the output is a first feature block corresponding to the unmasked image block; and determining the similarity between each aerial image to be detected based on each first image feature map. The application can significantly improve the accuracy of detecting the similarity of aerial images.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, electronic device, and storage medium for detecting the similarity of aerial images. Background Technology

[0002] With the development of unmanned aerial vehicles (UAVs), acquiring aerial imagery has become increasingly easier. However, the acquired imagery data is often disorganized, massive in quantity, and lacks a positioning and orientation system (POS) for many images. Therefore, retrieving similar images from these datasets has become a critical issue.

[0003] Aerial image similarity detection refers to finding similar images (images of the same area at different times and angles) among multiple aerial images, given a target image. Currently, existing image similarity detection algorithms, such as those based on word trees, require a large amount of storage space to store the word tree, while deep learning-based methods are mainly for natural images and are less suitable for aerial image similarity detection, resulting in low accuracy in aerial image similarity detection. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method, apparatus, electronic device and storage medium for detecting the similarity of aerial images, which can significantly improve the accuracy of detecting the similarity of aerial images.

[0005] In a first aspect, embodiments of the present invention provide a method for similarity detection of aerial images, comprising:

[0006] Acquire an aerial image set; wherein the aerial image set includes at least two aerial images to be detected;

[0007] The first image feature map of each of the aerial images to be detected is extracted using a pre-trained image feature extraction model. The image feature extraction model is trained using unlabeled aerial images. During the training phase, the input of the image feature extraction model is an unmasked image block in the unlabeled aerial image, and the output is the first feature block corresponding to the unmasked image block.

[0008] The similarity between each of the aerial images to be detected is determined based on each of the first image feature maps.

[0009] In one embodiment, the training step of the image feature extraction model includes:

[0010] Unlabeled aerial images are acquired, and the unlabeled aerial images are divided into blocks to obtain multiple aerial image blocks;

[0011] The aerial image block is divided into masked image blocks and unmasked image blocks;

[0012] The unmasked image block is input into a neural network to determine the recovered aerial image based on the unmasked image block; wherein, the neural network includes an encoder and a decoder;

[0013] The neural network is trained based on the unlabeled aerial image and the restored aerial image, and the encoder of the trained neural network is used as the image feature extraction model.

[0014] In one implementation, determining the restored aerial imagery based on the unmasked image blocks using the neural network includes:

[0015] The encoder extracts the first feature block corresponding to each unmasked image block;

[0016] A second image feature map corresponding to the unlabeled aerial image is constructed based on each of the first feature blocks;

[0017] The decoder determines the recovered aerial image based on the second image feature map.

[0018] In one implementation, constructing a second image feature map corresponding to the unlabeled aerial image based on each of the first feature blocks includes:

[0019] Determine the second feature block corresponding to each of the masked image blocks; wherein, the second feature block is a mask blank feature block;

[0020] Based on the positions of the masked image block and the unmasked image block in the unlabeled aerial image, the first feature block and the second feature block are stitched together to obtain the second image feature map corresponding to the unlabeled aerial image.

[0021] In one implementation, determining the similarity between each of the aerial images to be detected based on each of the first image feature maps includes:

[0022] Based on each of the first image feature maps, determine the normalized image features corresponding to each of the aerial images to be detected.

[0023] For any two aerial images to be detected, calculate the cosine distance between the normalized image features corresponding to the two aerial images to be detected, so as to obtain the similarity between the two aerial images to be detected.

[0024] In one implementation, based on each of the first image feature maps, determining the normalized image features corresponding to each of the aerial images to be detected includes:

[0025] For each of the first image feature maps, perform sum-pooling processing on each channel of the first image feature map to obtain the depth feature corresponding to each channel;

[0026] The depth features corresponding to each channel are normalized to determine the normalized image features corresponding to the first image feature map.

[0027] In one implementation, after determining the similarity between each of the aerial images to be detected based on each of the first image feature maps, the method further includes:

[0028] Based on the similarity between each of the aerial images to be detected, a similarity matrix corresponding to the aerial image set is constructed;

[0029] In this similarity matrix, both rows and columns represent the aerial images to be detected, and each element of the similarity matrix represents the similarity between the aerial images to be detected corresponding to that row and the aerial images to be detected corresponding to that column.

[0030] Secondly, embodiments of the present invention also provide an aerial image similarity detection device, comprising:

[0031] An image acquisition module is used to acquire an aerial image set; wherein the aerial image set includes at least two aerial images to be detected;

[0032] The feature extraction module is used to extract a first image feature map for each of the aerial images to be detected using a pre-trained image feature extraction model. The image feature extraction model is trained using unlabeled aerial images. During the training phase, the input of the image feature extraction model is an unmasked image block in the unlabeled aerial image, and the output is the first feature block corresponding to the unmasked image block.

[0033] A similarity detection module is used to determine the similarity between each of the aerial images to be detected based on each of the first image feature maps.

[0034] Thirdly, embodiments of the present invention also provide an electronic device, including a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the method described in any of the first aspects.

[0035] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the method described in any of the first aspects.

[0036] This invention provides a method, apparatus, electronic device, and storage medium for detecting the similarity of aerial images. After acquiring an aerial image set including at least two aerial images to be detected, a pre-trained image feature extraction model can be used to extract a first image feature map for each aerial image to be detected. Finally, the similarity between each aerial image to be detected is determined based on each first image feature map. The image feature extraction model is trained using unlabeled aerial images. During the training phase, the input to the image feature extraction model is an unmasked image block in the unlabeled aerial image, and the output is the first feature block corresponding to the unmasked image block. The image feature extraction model used in the above method is obtained through self-supervised training using unlabeled aerial images. During the training process, the mapping relationship between the unmasked image block and the first feature block is learned, thereby enabling the trained image feature extraction model to effectively extract the first image feature map of the aerial image to be detected. Based on this, the accuracy of detecting the similarity of aerial images can be significantly improved.

[0037] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0038] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0039] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating a method for detecting the similarity of aerial images according to an embodiment of the present invention.

[0041] Figure 2 A schematic diagram illustrating the training process of an image feature extraction model provided in an embodiment of the present invention;

[0042] Figure 3 A flowchart illustrating another aerial image similarity detection method provided in an embodiment of the present invention;

[0043] Figure 4 A flowchart illustrating another aerial image similarity detection method provided in an embodiment of the present invention;

[0044] Figure 5 A schematic diagram illustrating the reconstruction effect after masking an aerial image, provided in an embodiment of the present invention;

[0045] Figure 6 A schematic diagram of an aerial image set provided in an embodiment of the present invention;

[0046] Figure 7 A schematic diagram of a miou matrix provided in an embodiment of the present invention;

[0047] Figure 8 A schematic diagram of a similarity matrix provided in an embodiment of the present invention;

[0048] Figure 9 A schematic diagram of the structure of an aerial image similarity detection device provided in an embodiment of the present invention;

[0049] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] Currently, existing image similarity detection algorithms include traditional methods and deep learning-based methods. Traditional methods extract visual features from images using the SIFT (Scale Invariant Feature Transform) algorithm and then build an index (vocabulary tree) of a large number of features to retrieve similar images from a large-scale imagery dataset. However, this method requires building and storing the vocabulary tree, which consumes a significant amount of storage space. Deep learning-based methods extract image features through neural networks and then calculate the similarity between images using similarity operators (cosine similarity, Euclidean distance). However, the feature extraction network of this method is a network pre-trained on ImageNet, which differs significantly from aerial imagery. Therefore, this invention provides a method, apparatus, electronic device, and storage medium for aerial image similarity detection, which can significantly improve the accuracy of aerial image similarity detection.

[0052] To facilitate understanding of this embodiment, a detailed description of an aerial image similarity detection method disclosed in this invention will be provided first, see [link to relevant documentation]. Figure 1 The diagram shows a flowchart of a similarity detection method for aerial images. The method mainly includes the following steps S102 to S106:

[0053] Step S102: Obtain an aerial image set; wherein the aerial image set includes at least two aerial images to be detected, which are images acquired through photography by an aircraft. In one embodiment, an upload channel can be provided to the user, allowing the user to upload at least two aerial images to be detected.

[0054] Step S104: Extract the first image feature map for each aerial image to be detected using a pre-trained image feature extraction model. The image feature extraction model is trained using unlabeled aerial images, specifically through self-supervised training. During the training phase, the input to the image feature extraction model is an unmasked image block from the unlabeled aerial image, and the output is the first feature block corresponding to the unmasked image block. During the application phase, the input to the image feature extraction model is the aerial image to be detected, and the output is the first image feature map. In one embodiment, inputting the aerial image to be detected into the image feature extraction model yields the first image feature map output by the model.

[0055] Step S106: Determine the similarity between each aerial image to be detected based on each first image feature map. In one embodiment, each first image feature map can be subjected to pooling to obtain corresponding depth features. Then, the depth features can be normalized to obtain the normalized image features of the aerial image to be detected. Finally, the similarity between any two aerial images to be detected can be determined based on their normalized image features.

[0056] The aerial image similarity detection method provided in this embodiment of the invention uses an image feature extraction model that is obtained through self-supervised training using unlabeled aerial images. During the training process, the model learns the mapping relationship between unmasked image blocks and first feature blocks, thereby enabling the trained image feature extraction model to effectively extract the first image feature map of the aerial image to be detected. Based on this, the accuracy of detecting the similarity of aerial images can be significantly improved.

[0057] In practical applications, to enable the image feature extraction model to effectively extract features from aerial images, it is necessary to train the model. This invention provides an implementation method for training the image feature extraction model. (See attached document.) Figure 2 The diagram shown illustrates the training process of an image feature extraction model, including steps 1 to 4:

[0058] Step 1: Obtain unlabeled aerial images and divide them into blocks to obtain multiple aerial image blocks.

[0059] Step 2: Divide the aerial image blocks into masked image blocks and unmasked image blocks. In one implementation, a mask rate can be pre-configured, and aerial image blocks are randomly masked according to the mask rate to obtain masked image blocks. The aerial image blocks that are not masked are the unmasked image blocks. For example, assuming a mask rate of 75%, 75% of the aerial image blocks are randomly selected for masking, thereby dividing the aerial image blocks into 75% masked image blocks and 25% unmasked image blocks.

[0060] Step 3 involves inputting the unmasked image blocks into a neural network to determine the restored aerial image based on the unmasked image blocks; wherein the neural network includes an encoder and a decoder. In an optional implementation, the neural network can employ a ViT-B (Vision Transformer Base) model and be trained using a method of predicting masked image blocks (Mask Image Model, MIM), thereby enabling the neural network to learn effective features of the aerial image.

[0061] Specifically, this embodiment of the invention provides a specific implementation method for determining the restored aerial image based on unmasked image blocks using a neural network, as shown in steps 3.1 to 3.3 below:

[0062] Step 3.1: Extract the first feature block corresponding to each unmasked image block using an encoder. In one embodiment, the unmasked image block is input into an encoder, and the first feature block of the unmasked image block is obtained through the encoder.

[0063] Step 3.2: Construct a second image feature map corresponding to the unlabeled aerial image based on each first feature block. In one implementation, the second feature block corresponding to each masked image block is determined. Then, according to the positions of the masked image block and the unmasked image block in the unlabeled aerial image, the first feature block and the second feature block are stitched together to obtain the second image feature map corresponding to the unlabeled aerial image. The second feature block is a masked blank feature block. Please continue to the next step. Figure 2 Taking the masked image block in the first row and first column as an example, a mask blank feature block can be added at the corresponding position of the masked image block. Repeat this process to add a mask blank feature block at the corresponding position of each masked image block. Combined with the first feature block at the corresponding position of the unmasked image block, a complete second image feature map can be obtained.

[0064] Step 3.3: The restored aerial image is determined by the decoder based on the second image feature map. In one embodiment, the second image feature map is input into the decoder to obtain the restored image output by the decoder.

[0065] Step 4: Train the neural network based on the unlabeled aerial imagery and the restored aerial imagery, and use the encoder of the trained neural network as the image feature extraction model. In one implementation, the neural network can be trained by calculating the pixel value difference between the restored image and the original unlabeled aerial imagery, and this calculation can be performed only on the masked image patches. In a specific implementation, the Mean Square Error (MSE) loss function can be used, as shown below:

[0066]

[0067] Where Y i Y represents the original pixel value of the masked image block. i ′ This represents the pixel value at that location after reconstruction by the model, and Loss represents the loss value.

[0068] In the application phase, the trained neural network is pruned, removing the decoder part and retaining the encoder part, which is then used as the image feature extraction model. During the application phase, the aerial image to be detected is input into the image feature extraction model, which yields the first image feature map corresponding to the input aerial image. Subsequently, the first image feature map is normalized to obtain the corresponding feature vector (i.e., normalized image features).

[0069] This invention employs a self-supervised method to train an image feature extraction model. The training data does not require labeling, thus eliminating the cumbersome labeling process and simplifying training. Furthermore, large datasets can be easily processed (by filling and cropping large images) before training. By training on large datasets, this invention also enhances the feature extraction performance of the image feature extraction model.

[0070] Based on the foregoing embodiments, this invention provides a specific implementation of an aerial image similarity detection method, see [link to relevant documentation]. Figure 3 The flowchart of another aerial image similarity detection method is shown. The aerial image to be detected is input into the image feature extraction model (i.e., the encoder mentioned above). Then, the image feature extraction model extracts a first image feature map of the aerial image to be detected, and the similarity between each aerial image to be detected is determined based on each first image feature map. When performing the step of determining the similarity between each aerial image to be detected, see steps a to b below:

[0071] Step a: Based on each first image feature map, determine the normalized image features corresponding to each aerial image to be detected. In specific implementation, for each first image feature map, sum-pooling can be performed on each channel of the first image feature map to obtain the depth features corresponding to each channel, and then the depth features corresponding to each channel can be normalized to determine the normalized image features corresponding to the first image feature map.

[0072] For example, assume σ∈R C×W×H Let C be the feature map output by the feature network, where C is the number of channels, and W and H are the width and height of the feature map, respectively. If pooling is used as the depth feature of the first image feature map, then the image feature of the c-th channel can be defined as:

[0073]

[0074] In practical implementation, the pooling formula described above can be used to perform pooling calculations on all channels of the feature map to obtain the depth feature F:

[0075]

[0076] Then, by normalizing the depth feature F, we can obtain the normalized image feature F′:

[0077]

[0078] Where V c F′ represents the feature of the c-th channel after normalization, and F′ is the feature vector (i.e., normalized image feature) obtained by transforming the first image feature map.

[0079] Step b: For any two aerial images to be detected, calculate the cosine distance between the normalized image features corresponding to the two aerial images to obtain the similarity between the two aerial images. In one embodiment, the similarity calculation is performed on the normalized image features obtained by the image feature extraction module, and cosine similarity can be selected. Assume that the normalized image features obtained by the neural network for two aerial images i,j to be detected are F i ,F j Then the cosine similarity S between the two aerial images to be detected i,j for:

[0080]

[0081] In the above formula, θ is the angle between the two vectors.

[0082] In one implementation, after determining the similarity between each aerial image to be detected, a similarity matrix corresponding to the aerial image set can be further constructed. Specifically, the similarity matrix can be constructed based on the similarity between each aerial image to be detected. Here, both rows and columns of the similarity matrix represent aerial images to be detected, and each element of the similarity matrix represents the similarity between the aerial image to be detected in that row and the aerial image to be detected in that column. For example, assuming the aerial image set includes aerial image 1, aerial image 2, and aerial image 3 to be detected, the similarity matrix would have three rows and three columns. The first row and second column represent the similarity between aerial image 1 and aerial image 2, and the first row and third column represent the similarity between aerial image 1 and aerial image 3. Similarly, the similarity matrix can reflect the similarity between each aerial image to be detected.

[0083] In summary, the aerial image similarity detection method provided by this invention, by using a transformer as the backbone network and designing an image feature extraction model, can fully utilize a large number of unlabeled aerial images for self-supervised training. The trained image feature extraction model can effectively extract aerial image features, demonstrating wide applicability and high accuracy in aerial image similarity detection. This invention can provide fundamental technical support for fields such as aerial image similarity detection and 3D reconstruction.

[0084] For ease of understanding, this invention also provides another method for similarity detection of aerial images, see [link to relevant documentation]. Figure 4 The flowchart of another aerial image similarity detection method is shown, which includes a training phase and an application phase. The training phase includes aerial image data acquisition, aerial image data processing, and self-supervised training of the image feature extraction model. The application phase includes image feature extraction model acquisition, aerial image data (UVA) acquisition, and similarity detection.

[0085] This invention provides an application example of an aerial image similarity detection method. First, see... Figure 5 The diagram shown illustrates the reconstruction effect after masking an aerial image. It can be seen that the image feature extraction model can effectively reconstruct the masked image block, indicating that the image feature extraction model has effectively learned the relevant features of the aerial image.

[0086] In the aerial image similarity assessment, a set of 3605 aerial images requiring similarity detection was identified. Some of these aerial images are listed below, such as... Figure 6 The diagram shows a set of aerial images that has been normalized, with each image being 1000*1000 pixels.

[0087] The size of the overlapping region of these images is obtained through 3D reconstruction, and its miou is calculated as a similarity index (ground truth). The resulting miou matrix is ​​as follows: Figure 7 The diagram shows a miou matrix, where brighter areas indicate higher similarity between two aerial images. The bright lines on the diagonal represent the similarity between the aerial image and itself, with a value of 1. If two aerial images have no similarity, the value is 0. Feature vectors are obtained through a feature extraction network, and their similarity matrix is ​​calculated, such as... Figure 8 The diagram shows a similarity matrix.

[0088] Compare Figure 7 and Figure 8As can be seen, the brightness and darkness of the two images are basically similar, indicating that the embodiment of the present invention can extract the features of aerial images relatively well. Furthermore, the accuracy of the 10 images with the highest similarity was calculated, and the embodiment of the present invention can achieve an accuracy of 90%. The accuracy of the 15 images with the highest similarity can reach 83.49%.

[0089] Regarding the aerial image similarity detection method provided in the foregoing embodiments, this invention provides an aerial image similarity detection device, see [link to related documentation]. Figure 9 The diagram shows a structural schematic of an aerial image similarity detection device, which mainly includes the following parts:

[0090] The image acquisition module 902 is used to acquire an aerial image set, wherein the aerial image set includes at least two aerial images to be detected;

[0091] The feature extraction module 904 is used to extract the first image feature map of each aerial image to be detected through a pre-trained image feature extraction model. The image feature extraction model is trained on unlabeled aerial images. During the training phase, the input of the image feature extraction model is an unmasked image block in the unlabeled aerial image, and the output is the first feature block corresponding to the unmasked image block.

[0092] The similarity detection module 906 is used to determine the similarity between each aerial image to be detected based on each first image feature map.

[0093] The aerial image similarity detection device provided in this embodiment of the invention uses an image feature extraction model that is obtained through self-supervised training using unlabeled aerial images. During the training process, the model learns the mapping relationship between unmasked image blocks and first feature blocks, thereby enabling the trained image feature extraction model to effectively extract the first image feature map of the aerial image to be detected. Based on this, the accuracy of detecting the similarity of aerial images can be significantly improved.

[0094] In one implementation, a training module is also included, for:

[0095] Acquire unlabeled aerial images and divide them into blocks to obtain multiple aerial image blocks;

[0096] The aerial image block is divided into masked image blocks and unmasked image blocks;

[0097] Unmasked image blocks are input into a neural network to determine the restored aerial image based on the unmasked image blocks; the neural network includes an encoder and a decoder.

[0098] The neural network was trained based on unlabeled aerial imagery and restored aerial imagery, and the encoder of the trained neural network was used as the image feature extraction model.

[0099] In one implementation, the training module is further used for:

[0100] The encoder extracts the first feature block corresponding to each unmasked image block;

[0101] Construct a second image feature map corresponding to the unlabeled aerial image based on each first feature block;

[0102] The restored aerial imagery is determined by the decoder based on the second image feature map.

[0103] In one implementation, the training module is further used for:

[0104] Determine the second feature block corresponding to each masked image block; wherein, the second feature block is the mask blank feature block;

[0105] Based on the positions of the masked image block and the unmasked image block in the unlabeled aerial image, the first feature block and the second feature block are stitched together to obtain the second image feature map corresponding to the unlabeled aerial image.

[0106] In one implementation, the similarity detection module 906 is further configured to:

[0107] Based on each first image feature map, determine the normalized image features corresponding to each aerial image to be detected.

[0108] For any two aerial images to be detected, calculate the cosine distance between the normalized image features corresponding to the two aerial images to obtain the similarity between the two aerial images.

[0109] In one implementation, the similarity detection module 906 is further configured to:

[0110] For each first image feature map, perform sum-pooling on each channel of the first image feature map to obtain the depth feature corresponding to each channel;

[0111] The depth features corresponding to each channel are normalized to determine the normalized image features corresponding to the first image feature map.

[0112] In one implementation, a matrix construction module is also included, for:

[0113] Based on the similarity between each aerial image to be detected, a similarity matrix corresponding to the aerial image set is constructed;

[0114] In this similarity matrix, both rows and columns represent aerial images to be detected, and each element of the similarity matrix represents the similarity between the aerial image to be detected in that row and the aerial image to be detected in that column.

[0115] The device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0116] This invention provides an electronic device, specifically, the electronic device includes a processor and a storage device; the storage device stores a computer program, and the computer program, when run by the processor, executes the method described in any of the above embodiments.

[0117] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device 100 includes: a processor 10, a memory 11, a bus 12 and a communication interface 13. The processor 10, the communication interface 13 and the memory 11 are connected through the bus 12. The processor 10 is used to execute executable modules, such as computer programs, stored in the memory 11.

[0118] The memory 11 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 13 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0119] Bus 12 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 10 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0120] The memory 11 is used to store programs. After receiving an execution instruction, the processor 10 executes the programs. The method executed by the device for defining the flow process disclosed in any of the foregoing embodiments of the present invention can be applied to the processor 10 or implemented by the processor 10.

[0121] Processor 10 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 10 or by instructions in software form. Processor 10 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 11. The processor 10 reads the information in memory 11 and, in conjunction with its hardware, completes the steps of the above method.

[0122] The computer program product of the readable storage medium provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the foregoing method embodiments. For specific implementation, please refer to the foregoing method embodiments, which will not be repeated here.

[0123] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0124] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An aerial image similarity detection method, characterized by, The method comprises the following steps: acquiring an aerial image set; wherein the aerial image set comprises at least two aerial images to be detected; extracting a first image feature map of each of the aerial images to be detected through a pre-trained image feature extraction model; wherein the image feature extraction model is trained through unlabeled aerial images, and in the training stage, the input of the image feature extraction model is an unmasked image block in the unlabeled aerial images, and the output is a first feature block corresponding to the unmasked image block; determining the similarity between each of the aerial images to be detected based on each of the first image feature maps, comprising: for each of the first image feature maps, performing sum-pooling processing on each channel of the first image feature map to obtain a depth feature corresponding to each of the channels; performing normalization processing on the depth feature corresponding to each of the channels to determine a normalized image feature corresponding to the first image feature map; for any two of the aerial images to be detected, calculating the cosine distance between the normalized image features corresponding to the two aerial images to be detected to obtain the similarity between the two aerial images to be detected; the training steps of the image feature extraction model comprise: acquiring an unlabeled aerial image, and performing block processing on the unlabeled aerial image to obtain a plurality of aerial image blocks; dividing the aerial image blocks into masked image blocks and unmasked image blocks; inputting the unmasked image blocks into a neural network to determine a restored aerial image based on the unmasked image blocks through the neural network; wherein the neural network comprises an encoder and a decoder, and the neural network adopts a ViT-B model; training the neural network based on the unlabeled aerial image and the restored aerial image using a mean square error loss function, and using the encoder of the trained neural network as an image feature extraction model; determining a restored aerial image based on the unmasked image blocks through the neural network, comprising: extracting a first feature block corresponding to each of the unmasked image blocks through the encoder; constructing a second image feature map corresponding to the unlabeled aerial image based on each of the first feature blocks; determining a restored aerial image based on the second image feature map through the decoder; constructing a second image feature map corresponding to the unlabeled aerial image based on each of the first feature blocks, comprising: determining a second feature block corresponding to each of the masked image blocks; wherein the second feature block is a masked blank feature block; splicing the first feature block and the second feature block according to the positions of the masked image blocks and the unmasked image blocks in the unlabeled aerial image to obtain a second image feature map corresponding to the unlabeled aerial image.

2. The similarity detection method of aerial images according to claim 1, characterized in that, After determining the similarity between each of the aerial images to be detected based on each of the first image feature maps, the method further comprises: constructing a similarity matrix corresponding to the aerial image set based on the similarity between each of the aerial images to be detected; The rows and columns of the similarity matrix represent the to-be-detected aerial images, and each element of the similarity matrix represents the similarity between the to-be-detected aerial image corresponding to the row and the to-be-detected aerial image corresponding to the column.

3. An aerial image similarity detection device, characterized by comprising: Comprise: An image acquisition module configured to acquire a set of aerial images, wherein the set of aerial images comprises at least two to-be-detected aerial images; A feature extraction module configured to extract, by using a pre-trained image feature extraction model, a first image feature map of each of the to-be-detected aerial images, wherein the image feature extraction model is trained by using unlabeled aerial images, and in a training stage, an input of the image feature extraction model is an unmasked image block in the unlabeled aerial images, and an output of the image feature extraction model is a first feature block corresponding to the unmasked image block; A similarity detection module configured to determine, based on each of the first image feature maps, a similarity between each of the to-be-detected aerial images, comprising: for each of the first image feature maps, performing sum-pooling processing on each channel of the first image feature map to obtain a depth feature corresponding to each of the channels; performing normalization processing on the depth feature corresponding to each of the channels to determine a normalized image feature corresponding to the first image feature map; and for any two of the to-be-detected aerial images, calculating a cosine distance between the normalized image features corresponding to the two to-be-detected aerial images to obtain a similarity between the two to-be-detected aerial images; Further comprising a model training module configured to: acquire an unlabeled aerial image, and perform block processing on the unlabeled aerial image to obtain a plurality of aerial image blocks; divide the aerial image blocks into masked image blocks and unmasked image blocks; input the unmasked image blocks into a neural network to determine, by using the neural network, a restored aerial image based on the unmasked image blocks; wherein the neural network comprises an encoder and a decoder, and the neural network adopts a ViT-B model; train the neural network based on the unlabeled aerial image and the restored aerial image by using a mean square error loss function, and use the encoder of the trained neural network as an image feature extraction model; The model training module is specifically configured to: extract, by using the encoder, a first feature block corresponding to each of the unmasked image blocks; construct a second image feature map corresponding to the unlabeled aerial image based on each of the first feature blocks; and determine, by using the decoder, a restored aerial image based on the second image feature map; The model training module is specifically configured to: determine a second feature block corresponding to each of the masked image blocks, wherein the second feature block is a masked blank feature block; and splice the first feature blocks and the second feature blocks according to positions of the masked image blocks and the unmasked image blocks in the unlabeled aerial image to obtain a second image feature map corresponding to the unlabeled aerial image.

4. An electronic device, comprising: A processor and a memory, the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method in any one of claims 1 to 2.

5. A computer readable storage medium, characterized in that, The computer readable storage medium stores computer executable instructions that, when invoked and executed by the processor, cause the processor to implement the method of any one of claims 1 to 2.

Images