Visual prompt guided remote sensing image similar target detection method and device

Abstract

The invention discloses a visual prompt guided remote sensing image similar target detection method and device. The method comprises the following steps: selecting and preprocessing remote sensing data, and performing data enhancement on images and visual prompts to obtain a preprocessed data set; constructing a target detection model; using a CNN backbone network to perform feature extraction on the preprocessed image, and inputting the extracted image into an encoder of a Transform; carrying out feature extraction on the visual prompt by using a CNN backbone network, adding the extracted visual prompt and a query object, and inputting the added visual prompt and the query object into a Transform decoder; the preprocessed image and the processing data of the visual prompt are fused in a decoder, a bounding box and a label are output through an FNN, and a prediction result is obtained; and training the network by using the loss function and the prediction result to obtain a final target detection model. According to the method, a multi-classification task is converted into a dichotomy task (matched or mismatched), an adopted remote sensing data set is processed, and Visual Prompt is introduced, so that a DETR model is expanded to a similar target detection task, and the model performance is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of remote sensing image application, and relates to a remote sensing image similar target detection method guided by visual cues, and specifically to a method for automatically generating positions of all targets with the same semantic category as the visual cues from remote sensing images. Background Art

[0002] Object detection in remote sensing images is an important application in the field of remote sensing. Its main task is to automatically detect objects of interest in remote sensing images, such as buildings, roads, water bodies, etc. It provides important data support and application basis for geographic information systems, urban planning, environmental monitoring and other fields, which is of great significance.

[0003] With the rapid development of artificial intelligence and hardware-related technologies, deep learning technologies represented by the FasterR-CNN network have achieved great success in the field of target detection. FasterR-CNN uses the RPN network to extract candidate boxes, and then classifies and regresses these candidate boxes based on the feature extraction network to obtain prediction results (Reference 1). The FasterR-CNN model has high accuracy, good robustness, and superior performance, but it uses NMS to avoid overlapping boxes and uses classification scores as screening criteria, which is not particularly friendly to occluded objects. Secondly, the FasterR-CNN model needs to generate a large number of anchors for the subsequent selection of positive and negative samples.

[0004] In 2017, Google proposed the Transformer model in the paper "Attention is All you need" (Reference 2), which uses the self-attention mechanism to replace the RNN network structure commonly used in NLP tasks. It can make full use of GPU resources in parallel computing and has good performance in capturing global information. DETR applies Transformer to the target detection task of images for the first time. It does not need to use NMS or generate anchors, but directly performs end-to-end set prediction. Its performance is comparable to Faster R-CNN, opening up a new technical route for image target detection tasks (Reference 3). However, DETR is only used in target detection tasks and is difficult to be directly applied to similar target detection tasks of remote sensing images. At present, there is still a lack of a technical solution that can directly apply the DETR model to similar target detection tasks of remote sensing images.

[0005] References:

[0006] [Document 1] Ren, S., He, K., Girshick, RB, Sun, J.: Faster R-CNN: Towards real-time object detection with region proposal networks. PAMI (2015).

[0007] [Document 2]Vaswani,A.,Shazeer,N.,Parmar,N.,Uszkoreit,J.,Jones,L.,Gomez,AN,Kaiser,L.,Polosukhin,I.:Attention is all you need.In:NeurIPS(2017).

[0008] [Document 3] Z.Dai, B.Cai, Y.Lin and J.Chen, "UP-DETR: Unsupervised Pre-training for ObjectDetection with Transformers," 2021 IEEE / CVF Conference on Computer Vision and PatternRecognition(CVPR), Nashville, TN, USA, 2021, pp.1601-1610, doi:10.1109 / CVPR46437.2021.00165. Summary of the invention

[0009] In view of the above technical problems, the present invention provides a method and device for detecting similar targets in remote sensing images guided by visual prompts. The method converts a multi-classification task into a binary classification task (match or mismatch), processes the adopted remote sensing dataset, and introduces a visual prompt, thereby extending the DETR model to the similar target detection task and improving the model performance.

[0010] The technical solution adopted by the present invention is as follows:

[0011] In a first aspect, a method for detecting similar targets in remote sensing images guided by visual cues is provided, comprising the following steps:

[0012] Remote sensing data selection and preprocessing to obtain preprocessed image data and preprocessed visual cue data;

[0013] Constructing a target detection model; the target detection model consists of a CNN backbone network, a Transformer and a loss function; the Transformer includes an encoder, a decoder and a feedforward neural network FNN;

[0014] The CNN backbone network is used to extract features from the preprocessed image data and input into the Transformer encoder. The CNN backbone network is used to extract features from the preprocessed visual cue data and then added to the query object and input into the Transformer decoder. The preprocessed image and visual cue processing data are fused in the decoder, and then the bounding box and label are output by the FNN to obtain the prediction result.

[0015] The network is trained using the loss function and prediction results to obtain the final target detection model.

[0016] In a possible implementation manner, the method for selecting and preprocessing remote sensing data to obtain preprocessed image data and preprocessed visual cue data includes:

[0017] Label the remote sensing data to obtain the category, width, height, and center point coordinate information of each target in the image and organize them into a dictionary;

[0018] The categories of targets in remote sensing data are divided into seen classes and unseen classes, and only the visible classes are retained;

[0019] Randomly select a target from the reserved remote sensing data as a visual cue and set its label to the same category as the visual cue;

[0020] The images in the retained remote sensing data are regularized and randomly resized; the brightness, contrast, saturation, and hue of the visual cues are randomly adjusted, then converted to grayscale, and then Gaussian blurred and regularized.

[0021] In a possible implementation manner, the target detection model is composed of a main network, a branch network and a loss function;

[0022] Among them, the main network is composed of CNN backbone network, encoder, decoder and FNN connection in sequence;

[0023] The branch network is composed of a CNN backbone network and a pooling layer in sequence; the branch network is connected to the decoder of the main network;

[0024] The CNN backbone network of the main network and the CNN backbone network of the branch network are the same network and share parameters.

[0025] Furthermore, the pre-processed image and the processed data of the visual cue are fused in the decoder by:

[0026] After the preprocessed visual cue data passes through the CNN backbone network and the pooling layer, it is added to the query object and then passes through the multi-head attention mechanism layer and the normalization layer to output the attention vector Query;

[0027] The preprocessed image data passes through the CNN backbone network and encoder to output a set of attention vectors Key and Value;

[0028] Query, Key, and Value are input into the multi-head attention mechanism layer and normalization layer to achieve data fusion.

[0029] In a possible implementation manner, the loss function is expressed as:

[0030]

[0031] Among them, y represents the true value; represents the predicted value; Indicates the binary classification result. If it is 0, it means the same as the visual cue category, and if it is 1, it means it is different; Represents the prediction box result; represents the predicted matching order obtained using the Hungarian algorithm; c i represents the true value of the i-th target category, i represents the i-th target in the image; b i Represents the true value of the i-th target position; Indicates the weight corresponding to the i-th target category; represents the cross entropy loss; represents the combination of L1 loss and GIoU loss, Indicates the judgment condition c i =0 is true.

[0032] In a possible implementation manner, the training method is:

[0033] The prediction results are combined with the true value of the bounding box and the label to calculate the loss function, and the gradient of the target detection model parameters is calculated, and the weights of the target detection model are updated according to the gradient.

[0034] In a second aspect, a remote sensing image similar target detection device guided by visual cues is provided, comprising:

[0035] Data acquisition and preprocessing module, used for remote sensing data selection and preprocessing, obtaining preprocessed image data and preprocessed visual prompt data;

[0036] A model building module, used to build a target detection model; the target detection model consists of a CNN backbone network, a Transformer and a loss function; the Transformer includes an encoder, a decoder and a feedforward neural network FNN;

[0037] The data fusion and output module is used to extract features from the preprocessed image data using the CNN backbone network and input it into the Transformer encoder; the preprocessed visual cue data is extracted using the CNN backbone network and then added to the query object and input into the Transformer decoder; the preprocessed image and visual cue processing data are fused in the decoder, and then the bounding box and label are output by the FNN to obtain the prediction result;

[0038] The final model acquisition module is used to train the network using the loss function and prediction results to obtain the final target detection model.

[0039] According to a third aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for detecting similar targets in remote sensing images guided by visual cues as described above is implemented.

[0040] In a fourth aspect, a non-transitory computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for detecting similar targets in remote sensing images guided by visual cues as described above is implemented.

[0041] In a fifth aspect, a computer program product is provided, comprising a computer program, characterized in that when the computer program is executed by a processor, the method for detecting similar targets in remote sensing images guided by visual cues as described above is implemented.

[0042] The present invention can make full use of the superiority of the DETR model in target detection, apply it to similar target detection, and introduce a visual cue to enable it to meet the needs of similar target detection tasks in remote sensing images. The beneficial effects are:

[0043] (1) Visual prompts are introduced into the DETR model, which effectively improves the model performance. After the visual prompts are processed by the CNN backbone network and global average pooling GAP, they are added to all query objects to achieve the fusion of image data and visual prompt data, and then input into the decoder together, making it easier for the model to predict objects with the same semantic category as the visual prompts.

[0044] (2) The DETR model is extended from the target detection task to the similar target detection task, which involves classification problems and data set processing. Specifically, the multi-classification problem of target detection is transformed into a binary classification problem of similar target detection, that is, predicting whether the classification result matches the visual cue, and applying it to the subsequent loss calculation and classification result output; for the processing of the data set, a target is randomly selected from all visible category targets as a visual cue, and finally only the targets with the same category as the visual cue are retained. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram of a neural network model for detecting similar targets in remote sensing images according to an embodiment of the present invention;

[0046] Figure 2 is an overall flow chart of an embodiment of the present invention;

[0047] Figure 3 A schematic diagram of the structure of a remote sensing image similar target detection device guided by visual cues provided by the present invention;

[0048] Figure 4 This is a schematic structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0049] In order to facilitate ordinary technicians in the field to understand and implement the present invention, the present invention is further described below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0050] Please see Figure 1 , a method for detecting similar targets in remote sensing images guided by visual cues provided by the present invention comprises the following steps:

[0051] Step 100: remote sensing data set selection and preprocessing, obtaining preprocessed image data and preprocessed visual cue data.

[0052] In this example, the CrowdAI Mapping Challenge dataset was selected for training and testing. This dataset provides the segmented data of remote sensing satellite RGB images and the corresponding location information of all buildings in the image. Each image contains a large number of objects of the same category, with perfect annotations and uniform image quality, which can be well trained and verified for similar target detection tasks of remote sensing images.

[0053] The dataset contains the following parts:

[0054] (1) The training set is a set of 280,741 satellite images (300x300 pixel RGB images. The test set is a set of 60,317 satellite images (300x300 pixel RGB images. The validation set provides 60,697 satellite images (300x300 pixel RGB images).

[0055] (2) The training set and test set provide annotation files in MS-COCO format, including image size image_size and area size, object category label label and corresponding position bounding box and other data.

[0056] As a specific embodiment, the remote sensing data selection and preprocessing, and the method for data enhancement of images and visual cues include:

[0057] Label the remote sensing data to obtain the category, width, height, and center point coordinate information of each target in the image and organize them into a dictionary;

[0058] The target categories in the remote sensing data are divided into seen categories and unseen categories, and the seen categories are screened and retained to obtain the retained remote sensing data;

[0059] Randomly select a target from the retained remote sensing data as a visual cue, and set its label to 0, indicating that it is of the same category as the visual cue;

[0060] The images in the retained remote sensing data are regularized and randomly resized; the brightness, contrast, saturation, and hue of the visual cues are randomly adjusted, then converted to grayscale, and then Gaussian blurred and regularized.

[0061] It should be noted that the similar object detection problem can be regarded as a binary classification problem, where 0 means the same category as the visual cue, and 1 means a different category from the visual cue.

[0062] The embodiment of the present invention selects all training set images from the data set for training, and uses the test set images for model evaluation. The remote sensing data set is read with images and annotation information, including image name image_id, image size image_size and area area, object category label label and corresponding position bounding box and other data, and organized into a dictionary for subsequent use. The target categories in the data set are divided into seen class and unseen class. For the training process, only the target objects of the seen class in the image are used, and all unseen classes are eliminated. In the verification process, all visible and unseen classes are used, and mAP is calculated according to the category, which is convenient for evaluating the performance of the similar target detection model. One of the remaining target objects is randomly selected as a visual prompt, and all target objects with different categories from the visual prompt are eliminated. At the same time, the labels of the remaining target objects are set to 1, and the final ground truth is the label (label=1) and bounding box of all objects with the same semantic category as the visual prompt. The reading of the data set images and annotations and the acquisition of the visual prompt VisualPrompt are successfully realized.

[0063] Data augmentation is performed on images and visual cues to increase the diversity of training samples and improve the robustness of the model. Images are regularized and randomly resized; visual cues are randomly adjusted in brightness, contrast, saturation, hue, converted to grayscale, Gaussian blurred, regularized, etc., and resized to (128,128).

[0064] It should be noted that the Visual Prompt is input into the decoder as the query object of DETR, which can serve as prompt information, so that the decoder can focus on the target with the same semantic category as the visual prompt. Combined with the global modeling capability of DETR, the task of similar target detection can be achieved.

[0065] Step 200: construct a target detection model; the target detection model consists of a CNN backbone network, a Transformer and a loss function; the Transformer includes an encoder, a decoder and a feedforward neural network FNN.

[0066] As a specific embodiment, the target detection model is composed of a main network, a branch network and a loss function; wherein the main network is sequentially composed of a CNN backbone network, an encoder, a decoder and an FNN connection; the branch network is sequentially composed of a CNN backbone network and a pooling layer; the branch network is connected to the decoder of the main network;

[0067] The CNN backbone network of the main network and the CNN backbone network of the branch network are the same network and share parameters.

[0068] Specifically, Figure 1 As shown in the figure, the main body of the basic network model consists of Transformer and CNN backbone network (CNNbackbone). Transformer adopts self-attention mechanism, which has good parallel computing performance and global information capture performance. In the model, Transformer adopts a 6-layer encoder and decoder architecture. CNN backbone network is used to extract features of images and visual cues.

[0069] As a specific embodiment, the calculation of the loss function includes two steps:

[0070] (1) Matching: Use the Hungarian algorithm to calculate the matching loss between the prediction and the ground truth to determine the optimal one-to-one matching order between the prediction and the ground truth.

[0071] (2) Calculating loss: Determine the one-to-one correspondence between predictions and true values After that, calculate the loss function. Hungarian loss includes and Two parts, It is the binary cross entropy loss function (BCE Loss, BinaryCross Entropy Loss), and the classification results are divided into matching (c i =0) and does not match (c i =1) Two categories, It is a combination of L1 loss and GIoU loss. L1 loss has different scale problems for target boxes of different sizes, while GIoU loss is scale invariant for target boxes of different sizes.

[0072] The loss function is:

[0073]

[0074] Among them, y represents the true value; represents the predicted value; Indicates the binary classification result. If it is 0, it means the same as the visual cue category, and if it is 1, it means it is different; Represents the prediction box result; represents the predicted matching order obtained using the Hungarian algorithm; c i represents the true value of the i-th target category, i represents the i-th target in the image; b i represents the true value of the i-th target position, Indicates the weight corresponding to the i-th target category; represents the cross entropy loss; represents the combination of L1 loss and GIoU loss, Indicates the judgment condition c i =0 is true.

[0075] Step 300: Use the CNN backbone network to extract features from the preprocessed image data and input it into the encoder of the Transformer; use the CNN backbone network to extract features from the preprocessed visual cue data and add it to the query object and input it into the decoder of the Transformer; the preprocessed image and visual cue processing data are fused in the decoder, and then the bounding box and label are output by the FNN to obtain the prediction result.

[0076] As a specific embodiment, step S300 includes the following sub-steps:

[0077] S310, using the CNN backbone network to extract compact feature representations from the preprocessed image, flattening it and adding it to the positional encoding before inputting it into the encoder. Each layer of the encoder includes a multi-head self-attention layer and a feed-forward neural network FNN. The encoder will output a set of attention vectors Key and Value, which will be used in the Multi-Head Attention layer of each decoder.

[0078] S320, the visual cue is added to all query objects after passing through the CNN backbone network and GAP (Global Average Pooling), so that the features of the visual cue are added to the query object queries, and then input into the decoder.

[0079] The data of S330, S310 and S320 are merged in the decoder.

[0080] Specifically, the fusion methods include:

[0081] After the preprocessed visual cue data passes through the CNN backbone network and the pooling layer, it is added to the query object and then passes through the multi-head attention mechanism layer and the normalization layer to output the attention vector Query;

[0082] The preprocessed image data passes through the CNN backbone network and encoder to output a set of attention vectors Key and Value;

[0083] Query, Key, and Value are input into the multi-head attention mechanism layer and normalization layer to achieve data fusion.

[0084] After data fusion, the visual cues will guide the decoder to find targets of the same category as the visual cues, so there is no need to rely on semantic information. Instead, the model is trained to learn the ability to detect similar targets using visual cues as cues.

[0085] The query object data of the backbone network is fused to make it easier for the model to predict objects with the same semantic category as the visual cues. The query object can be regarded as a learnable position encoding.

[0086] Step 400: Use the loss function and prediction results to train the network to obtain the final target detection model.

[0087] As a specific embodiment, after obtaining the prediction result, the loss function is calculated in combination with the true value (bounding box, label), and the gradient of the model parameters is calculated, and the network weights are updated according to the gradient.

[0088] After the network training is completed, the network weights are saved in the pth file. With the help of the pth file, similar target detection can be performed directly through the input remote sensing image and visual cues, and the final result can be output after visualization.

[0089] The following is a description of the remote sensing image similar target detection device guided by visual cues provided by the present invention. The remote sensing image similar target detection device guided by visual cues described below and the remote sensing image similar target detection method guided by visual cues described above can be referenced to each other.

[0090] Figure 3 is a schematic diagram of the structure of a remote sensing image similar target detection device guided by visual cues provided by an embodiment of the present invention, such as Figure 3 As shown, it includes: a data acquisition and preprocessing module 31, a model building module 32, a data fusion and output module 33 and a final model acquisition module 34, wherein:

[0091] The data acquisition and preprocessing module 31 is used for selecting and preprocessing remote sensing data, obtaining preprocessed image data and preprocessed visual prompt data;

[0092] A model building module 32 is used to build a target detection model; the target detection model is composed of a CNN backbone network, a Transformer and a loss function; the Transformer includes an encoder, a decoder and a feedforward neural network FNN;

[0093] The data fusion and output module 33 is used to extract features from the pre-processed image data using the CNN backbone network and input the features into the encoder of the Transformer; the pre-processed visual cue data is extracted using the CNN backbone network and then added to the query object and input into the decoder of the Transformer; the pre-processed image and the processed data of the visual cue are fused in the decoder, and then the bounding box and label are output by the FNN to obtain the prediction result;

[0094] The final model acquisition module 34 is used to train the network using the loss function and the prediction results to obtain the final target detection model.

[0095] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the remote sensing image similar target detection method guided by visual prompts.

[0096] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0097] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the remote sensing image similar target detection method guided by visual cues provided by the above-mentioned methods.

[0098] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the remote sensing image similar target detection method guided by visual cues provided by the above-mentioned methods.

[0099] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0100] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting similar targets in remote sensing images guided by visual cues, characterized in that: The following steps are involved: Remote sensing data selection and preprocessing to obtain preprocessed image data and preprocessed visual cue data; Constructing a target detection model; the target detection model consists of a CNN backbone network, a Transformer and a loss function; the Transformer includes an encoder, a decoder and a feedforward neural network FNN; The CNN backbone network is used to extract features from the preprocessed image data and input into the Transformer encoder. The CNN backbone network is used to extract features from the preprocessed visual cue data and then added to the query object and input into the Transformer decoder. The preprocessed image and visual cue processing data are fused in the decoder, and then the bounding box and label are output by the FNN to obtain the prediction result. The network is trained using the loss function and prediction results to obtain the final target detection model.

2. The method for detecting similar targets in remote sensing images guided by visual cues according to claim 1, characterized in that: The method for selecting and preprocessing remote sensing data to obtain preprocessed image data and preprocessed visual prompt data includes: Label the remote sensing data to obtain the category, width, height, and center point coordinate information of each target in the image and organize them into a dictionary; Classify the categories of targets in the remote sensing data into visible categories and invisible categories, filter and retain the visible categories, and obtain the retained remote sensing data; Randomly select a target from the reserved remote sensing data as a visual cue and set its label to the same category as the visual cue; The images in the retained remote sensing data are regularized and randomly resized; the brightness, contrast, saturation, and hue are randomly adjusted for visual cues, then converted to grayscale, and then Gaussian blurred and regularized.

3. The method for detecting similar targets in remote sensing images guided by visual cues according to claim 1, characterized in that: The target detection model consists of a main network, a branch network and a loss function; Among them, the main network is composed of CNN backbone network, encoder, decoder and FNN connection in sequence; The branch network is composed of a CNN backbone network and a pooling layer in sequence; the branch network is connected to the decoder of the main network; The CNN backbone network of the main network and the CNN backbone network of the branch network are the same network and share parameters.

4. The method for detecting similar targets in remote sensing images guided by visual cues according to claim 3, characterized in that: The pre-processed image and the processed data of the visual prompt are fused in the decoder in the following way: After the preprocessed visual cue data passes through the CNN backbone network and the pooling layer, it is added to the query object to obtain the first data set, and then passes through the multi-head attention mechanism layer and the normalization layer to output the attention vector Query; The preprocessed image data passes through the CNN backbone network and encoder to output a set of attention vectors Key and Value; Query, Key, and Value are input into the multi-head attention mechanism layer and normalization layer to achieve data fusion.

5. The method for detecting similar targets in remote sensing images guided by visual cues according to claim 1, characterized in that: The expression of the loss function is: Among them, y represents the true value; represents the predicted value; Indicates the binary classification result. If it is 0, it means the same as the visual cue category, and 0 means it is different; Represents the prediction box result; represents the predicted matching order obtained using the Hungarian algorithm; c i represents the true value of the i-th target category, i represents the i-th target in the image; b i Represents the true value of the i-th target position; Represents the weight coefficient corresponding to the true value of the i-th target category; represents the cross entropy loss; represents the combination of L1 loss and GIoU loss, Indicates the judgment condition c i =0 is true.

6. The method for detecting similar targets in remote sensing images guided by visual cues according to claim 1, characterized in that: The training method is: The prediction results are combined with the true value of the bounding box and the label to calculate the loss function, and the gradient of the target detection model parameters is calculated, and the weights of the target detection model are updated according to the gradient.

7. A remote sensing image similar target detection device guided by visual cues, characterized in that: include: Data acquisition and preprocessing module, used for remote sensing data selection and preprocessing, obtaining preprocessed image data and preprocessed visual prompt data; A model building module, used to build a target detection model; the target detection model consists of a CNN backbone network, a Transformer and a loss function; the Transformer includes an encoder, a decoder and a feedforward neural network FNN; The data fusion and output module is used to extract features from the preprocessed image data using the CNN backbone network and input it into the Transformer encoder; the preprocessed visual cue data is extracted using the CNN backbone network and then added to the query object and input into the Transformer decoder; The processed data of the preprocessed image and visual cues are fused in the decoder, and then the bounding box and label are output by the FNN to obtain the prediction result; The final model acquisition module is used to train the network using the loss function and prediction results to obtain the final target detection model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for detecting similar targets in remote sensing images guided by visual cues as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting similar targets in remote sensing images guided by visual cues as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for detecting similar targets in remote sensing images guided by visual cues as described in any one of claims 1 to 6 is implemented.

Images