Visual cue guided remote sensing image similar target detection method and device

Abstract

The application discloses a visual prompt guided remote sensing image similar target detection method and device. The method comprises the following steps: remote sensing data selection and preprocessing, data enhancement on images and visual prompts, and obtaining a preprocessed data set; a target detection model is constructed; a CNN backbone network is used for feature extraction on the preprocessed image and input into the encoder of the Transformer; the CNN backbone network is used for feature extraction on the visual prompt and added with a query object, and then input into the decoder of the Transformer; the processed data of the preprocessed image and the visual prompt are fused in the decoder, and then output through the FNN to obtain a bounding box and a label, and finally a prediction result is obtained; a loss function and the prediction result are used for network training, and finally a target detection model is obtained. The method converts a multi-classification task into a binary classification task (matching or not matching), processes a remote sensing data set, and introduces a visual prompt, so that the DETR model is expanded to a similar target detection task, and the model performance is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of remote sensing image application, and relates to a visual prompt guided remote sensing image similar target detection method, in particular to a method for automatically generating positions of all targets with the same semantic category as visual prompts from a remote sensing image. BACKGROUND

[0002] Target detection of remote sensing images is an important application in the field of remote sensing, and the main task is to automatically detect targets of interest in remote sensing images, such as buildings, roads, water bodies, etc., to provide important data support and application foundation for the fields of geographic information systems, urban planning, environmental monitoring, etc., and has significance.

[0003] With the rapid development of artificial intelligence and hardware related technologies, deep learning technologies represented by FasterR-CNN network have achieved great success in target detection field. FasterR-CNN uses RPN network to extract candidate frames, and then classifies and regresses these candidate frames based on a feature extraction network to obtain prediction results (Document 1). The FasterR-CNN model has high precision, good robustness and superior performance, but uses NMS to avoid overlapping frames and uses classification scores as a screening standard, which is not particularly friendly to occluded objects. In addition, the FasterR-CNN model needs to use a large number of anchor points to select positive and negative samples.

[0004] In 2017, Google proposed the Transformer model in the paper Attention is All You Need (Document 2), which uses a self-attention mechanism to replace the commonly used RNN network structure in NLP tasks, can parallel computing to fully utilize GPU resources, and has good performance in capturing global information. DETR first applies the Transformer to the target detection task of images, does not need to use NMS and does not need to generate anchor, but directly performs set prediction in an end-to-end manner, and can be compared with FasterR-CNN in performance, opening up a new technical route for image target detection tasks (Document 3). However, DETR is only used in the target detection task, and it is difficult to directly apply it to the similar target detection task of remote sensing images. At present, there is still a lack of a technical solution that can directly apply the DETR model to the similar target detection task of remote sensing images.

[0005] REFERENCES

[0006] [Document 1] Ren, S., He, K., Girshick, R. B., Sun, J.: Faster R-CNN: Towards real-time object detection with region proposal networks. PAMI (2015).

[0007] [Document 2] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: NeurIPS (2017).

[0008] [Document 3] Z. Dai, B. Cai, Y. Lin and J. Chen, "UP-DETR: Unsupervised Pre-training for Object Detection with Transformers," 2021 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2021, pp. 1601-1610, doi: 10.1109 / CVPR46437.2021.00165. SUMMARY

[0009] To solve the above technical problems, the present application provides a visual prompt guided remote sensing image similar target detection method and device. The method converts the multi-classification task into a binary classification task (match or not match), processes the remote sensing data set used, and introduces visual prompts, thereby expanding the DETR model to the similar target detection task and improving the model performance.

[0010] The technical solution adopted by the present application is as follows:

[0011] In a first aspect, a visual prompt guided remote sensing image similar target detection method is provided, comprising the following steps:

[0012] Remote sensing data selection and preprocessing to obtain preprocessed image data and preprocessed visual prompt data;

[0013] The target detection model is composed of a CNN backbone network, a Transformer, and a loss function. The Transformer includes an encoder, a decoder, and a feedforward neural network (FNN).

[0014] The preprocessed image data is extracted by the CNN backbone network and input into the encoder of the Transformer. The preprocessed visual cue data is extracted by the CNN backbone network and added to the query object, and then input into the decoder of the Transformer. The processed data of the preprocessed image and the visual cue are fused in the decoder, and then output the bounding box and the label by the FNN to obtain the prediction result.

[0015] The network is trained by using the loss function and the prediction result to obtain the final target detection model.

[0016] In one possible implementation, the method for selecting and preprocessing remote sensing data to obtain preprocessed image data and preprocessed visual cue data includes the following steps.

[0017] The remote sensing data is labeled to obtain the class, width, height, and center point coordinate information of each target in the image, which are organized into a dictionary.

[0018] The classes of the targets in the remote sensing data are divided into seen classes and unseen classes, and only the seen classes are retained.

[0019] A target is randomly selected from the retained remote sensing data as a visual cue, and the label of the target is set to the same class as the visual cue.

[0020] The images in the retained remote sensing data are normalized and randomly adjusted in size. The visual cue is randomly adjusted in brightness, contrast, saturation, and hue, and then converted into a grayscale image, followed by Gaussian blur and normalization.

[0021] In one possible implementation, the target detection model is composed of a main network, a branch network, and a loss function.

[0022] The main network is composed of a CNN backbone network, an encoder, a decoder, and a FNN in sequence.

[0023] The branch network is composed of a CNN backbone network and a pooling layer in sequence. The branch network is connected to the decoder of the main network.

[0024] The CNN backbone network of the main network and the CNN backbone network of the branch network are the same network and share parameters.

[0025] Further, the method for fusing the processed data of the preprocessed image and the visual cue in the decoder is as follows.

[0026] After the preprocessed visual cue data passes through the CNN backbone network and the pooling layer, it is added to the query object and then passes through the multi-head attention mechanism layer and the normalization layer to output the attention vector Query;

[0027] The preprocessed image data passes through the CNN backbone network and encoder to output a set of attention vectors Key and Value;

[0028] Query, Key, and Value are input into the multi-head attention mechanism layer and normalization layer to achieve data fusion.

[0029] In a possible implementation, the loss function is expressed as:

[0030]

[0031] Among them, y represents the true value; represents the predicted value; Indicates the binary classification result. If it is 0, it means the category is the same as the visual prompt, and if it is 1, it means it is different. Represents the prediction box result; Indicates the predicted matching order obtained using the Hungarian algorithm; c i represents the true value of the i-th target category, i represents the i-th target in the image; b i Represents the true value of the i-th target position; Indicates the weight corresponding to the i-th target category; represents the cross entropy loss; represents the union of L1 loss and GIoU loss, Indicates the judgment condition c i =0 is true.

[0032] In a possible implementation manner, the training method is:

[0033] The prediction results are combined with the true value of the bounding box and the label to calculate the loss function, and the gradient of the target detection model parameters is calculated, and the weights of the target detection model are updated according to the gradient.

[0034] In a second aspect, a device for detecting similar targets in remote sensing images guided by visual cues is provided, comprising:

[0035] Data acquisition and preprocessing module, used for remote sensing data selection and preprocessing, obtaining preprocessed image data and preprocessed visual cue data;

[0036] A model building module is used to build a target detection model; the target detection model consists of a CNN backbone network, a Transformer and a loss function; the Transformer includes an encoder, a decoder and a feedforward neural network FNN;

[0037] The data fusion and output module is used to extract features from preprocessed image data using the CNN backbone network and input them into the Transformer encoder. The preprocessed visual cue data is also extracted using the CNN backbone network and then added to the query object and input into the Transformer decoder. The preprocessed image and visual cue processing data are fused in the decoder and then output by the FNN with bounding boxes and labels to obtain the prediction results.

[0038] The final model acquisition module is used to train the network using the loss function and prediction results to obtain the final target detection model.

[0039] In a third aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for detecting similar targets in remote sensing images guided by visual cues as described above is implemented.

[0040] In a fourth aspect, a non-transitory computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for detecting similar targets in remote sensing images guided by visual cues as described above is implemented.

[0041] In a fifth aspect, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method for detecting similar targets in remote sensing images guided by visual cues as described above is implemented.

[0042] The present invention can fully utilize the superiority of the DETR model in target detection, apply it to similar target detection, and introduce a visual cue to enable it to meet the needs of similar target detection tasks in remote sensing images. Its beneficial effects are:

[0043] (1) Visual prompts are introduced into the DETR model, effectively improving model performance. After the visual prompts are processed by the CNN backbone network and global average pooling (GAP), they are added to all query objects to achieve the fusion of image data and visual prompt data. The data is then input into the decoder together, making it easier for the model to predict objects with the same semantic category as the visual prompts.

[0044] (2) The DETR model is extended from the target detection task to the similar target detection task, which involves classification problems and data set processing. Specifically, the multi-classification problem of target detection is transformed into a binary classification problem of similar target detection, that is, predicting whether the classification result matches the visual cue, and applying it to the subsequent loss calculation and classification result output; for the processing of the data set, a target is randomly selected from all visible category targets as a visual cue, and finally only the targets with the same category as the visual cue are retained. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Schematic diagram of a neural network model for detecting similar targets in remote sensing images according to an embodiment of the present invention;

[0046] Figure 2 is an overall flow chart of an embodiment of the present invention;

[0047] Figure 3 A schematic diagram of the structure of the device for detecting similar targets in remote sensing images guided by visual cues provided by the present invention;

[0048] Figure 4 This is a schematic structural diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0049] In order to facilitate ordinary technicians in this field to understand and implement the present invention, the present invention is further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the implementation examples described here are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0050] Please see Figure 1 , a method for detecting similar targets in remote sensing images using visual cues provided by the present invention, comprising the following steps:

[0051] Step 100: remote sensing data set selection and preprocessing to obtain preprocessed image data and preprocessed visual cue data.

[0052] In this example, the CrowdAI Mapping Challenge dataset was used for training and testing. This dataset provides segmented data for remote sensing satellite RGB images and the corresponding location information of all buildings in the images. Each image contains a large number of objects of the same category, with complete annotations and uniform image quality, making it an excellent training and validation task for similar object detection in remote sensing images.

[0053] The dataset contains the following parts:

[0054] (1) The training set is a set of 280741 satellite images (300x300 pixel RGB images. The test set is a set of 60317 satellite images (300x300 pixel RGB images. The validation set provides 60697 satellite images (300x300 pixel RGB images).

[0055] (2) The training set and test set provide annotation files in MS-COCO format, including image size image_size and size area, object class label lable and corresponding position bounding box data.

[0056] As a specific embodiment, the remote sensing data selection and preprocessing, the image and visual prompt data enhancement method comprises:

[0057] Labeling the remote sensing data to obtain the class, width, height, and center point coordinate information of each target in the image, and organizing them into a dictionary;

[0058] Divide the target classes in the remote sensing data into seen classes and unseen classes, filter and retain the seen classes, and obtain the retained remote sensing data;

[0059] Randomly select a target from the retained remote sensing data as a visual prompt, and set all its labels label to 0, indicating the same class as the visual prompt;

[0060] Regularize and randomly adjust the size of the images in the retained remote sensing data; randomly adjust the brightness, contrast, saturation, and hue of the visual prompt, then convert it to a grayscale image, and then perform Gaussian blur and regularization.

[0061] It should be noted that the similar target detection problem can be regarded as a binary classification problem, where 0 represents the same class as the visual prompt, and 1 represents a different class from the visual prompt.

[0062] The embodiment of the present application selects all training set images from the data set for training and uses test set images for model evaluation. The remote sensing data set reads the image and the annotation information, including the image name image_id, the image size image_size and the area area, the object class label lable and the corresponding position bounding box, and organizes a dictionary for subsequent use. The target classes in the data set are divided into visible classes seen class and invisible classes unsee class. For the training process, only the objects in the image are used for the seen class, and all unsee classes are removed, while in the validation process, all visible and invisible classes are used, and the mAP is calculated according to the class, which is convenient for evaluating the performance of the similar target detection model. One is randomly selected from all the remaining target objects as a visual prompt, and all target objects different from the visual prompt are removed, and the labels of the remaining target objects are all set to 1, and the final ground truth is the label (label = 1) and the bounding box of all objects with the same semantic category as the visual prompt. The reading of the data set image and the annotation is successfully realized, and the visual prompt VisualPrompt is obtained.

[0063] Data augmentation is performed on the image image and the visual prompt to increase the diversity of training samples and improve the robustness of the model. The image is regularized and the size is randomly adjusted; the visual prompt is randomly adjusted in brightness, contrast, saturation, hue, converted to a gray image, Gaussian blurred, regularized, etc., and the size is adjusted to (128, 128).

[0064] It should be noted that the visual prompt Visual Prompt is input into the decoder decoder of the DETR as a query object, which can serve as a prompt information, so that the decoder decoder focuses on the target with the same semantic category as the visual prompt, and adds the global modeling capability of the DETR, thereby realizing the similar target detection task.

[0065] Step 200: Construct a target detection model; the target detection model is composed of a CNN backbone network, a Transformer and a loss function; the Transformer includes an encoder, a decoder and a feedforward neural network FNN.

[0066] As a specific embodiment, the target detection model is composed of a main network, a branch network and a loss function; wherein the main network is composed of a CNN backbone network, an encoder, a decoder and an FNN connection in sequence; the branch network is composed of a CNN backbone network and a pooling layer in sequence; the branch network is connected to the decoder of the main network;

[0067] The CNN backbone network of the main network and the CNN backbone network of the branch network are the same network, sharing parameters.

[0068] Specifically, as shown in Figure 1 The basic network model body is composed of a Transformer and a CNN backbone network (CNN backbone). The Transformer adopts a self-attention mechanism, has good parallel computing performance and good performance in capturing global information. The Transformer in the model adopts a 6-layer encoder Encoder and decoder Decoder architecture. The CNN backbone network is used to extract the features of images and visual cues.

[0069] As a specific embodiment, the calculation of the loss function includes two steps:

[0070] (1) Matching: using the Hungarian algorithm (Hungarian algorithm) to calculate the matching loss between the prediction and the ground truth, to determine the optimal one-to-one matching order between the prediction and the ground truth

[0071] (2) Calculate the loss: after determining the one-to-one correspondence order between the prediction and the ground truth , calculate the loss function. The Hungarian loss includes and two parts, is a binary cross entropy loss function (BCE Loss, Binary Cross Entropy Loss), and the classification result is divided into matching (c i = 0) and non-matching (c i = 1) two categories, is the combination of L1 loss and GIoU loss, and the L1 loss has different scales for different size target boxes, while the GIoU loss has scale invariance for different size target boxes.

[0072] The loss function is:

[0073]

[0074] Wherein, y represents the true value; represents the predicted value; Indicates the binary classification result. If it is 0, it means the category is the same as the visual prompt, and if it is 1, it means it is different. Represents the prediction box result; Indicates the predicted matching order obtained using the Hungarian algorithm; c i represents the true value of the i-th target category, i represents the i-th target in the image; b i represents the true value of the i-th target position, Indicates the weight corresponding to the i-th target category; represents the cross entropy loss; represents the union of L1 loss and GIoU loss, Indicates the judgment condition c i =0 is true.

[0075] Step 300: Use the CNN backbone network to extract features from the preprocessed image data and input it into the Transformer encoder; use the CNN backbone network to extract features from the preprocessed visual cue data, add it to the query object, and input it into the Transformer decoder; the preprocessed image and visual cue processing data are fused in the decoder, and then the FNN outputs the bounding box and label to obtain the prediction result.

[0076] As a specific embodiment, step S300 includes the following sub-steps:

[0077] S310 uses a CNN backbone network to extract a compact feature representation from the preprocessed image. This feature representation is flattened and added to the positional encoding before being input into the encoder. Each encoder layer includes a multi-head self-attention layer and a feed-forward neural network (FNN). The encoder outputs a set of attention vectors, Key and Value, which are used in the multi-head attention layer of each decoder.

[0078] In step S320 , the visual cue is added to all query objects after passing through the CNN backbone network and GAP (Global Average Pooling), thereby adding the features of the visual cue to the query object queries, which are then input into the decoder.

[0079] The data of S330, S310 and S320 are fused in the decoder.

[0080] Specifically, the fusion methods include:

[0081] The preprocessed visual prompt data is added to the query object after passing through the CNN backbone network and the pooling layer, and then passes through the multi-head attention mechanism layer and the normalization layer to output an attention vector Query.

[0082] The preprocessed image data passes through the CNN backbone network and the encoder to output a set of attention vectors Key and Value.

[0083] The Query, Key and Value are input into the multi-head attention mechanism layer and the normalization layer to realize data fusion.

[0084] After data fusion, the visual prompt guides the decoder Decoder to find the target of the same category as the visual prompt, so that the model does not need to be trained according to the semantic information, but uses the visual prompt as the prompt information to learn the ability of detecting similar targets.

[0085] The data fusion of the query object of the backbone network makes it easier for the model to predict the object of the same category as the visual prompt in terms of semantics. The query object can be regarded as a learnable position encoding.

[0086] Step 400: training the network by using the loss function and the prediction result to obtain a final target detection model.

[0087] As a specific embodiment, after obtaining the prediction result, the loss function is calculated in combination with the true value (bounding box, label), the gradient of the model parameter is calculated, and the network weight is updated according to the gradient.

[0088] After the network training is completed, the network weight is kept in the pth file, and the pth file can be directly used to perform similar target detection on the input remote sensing image and the visual prompt, and the final result can be output after visualization.

[0089] The visual prompt guided remote sensing image similar target detection device provided by the present application is described below. The visual prompt guided remote sensing image similar target detection device described below can be correspondingly referred to the visual prompt guided remote sensing image similar target detection method described above.

[0090] Figure 3 is a structural schematic diagram of the visual prompt guided remote sensing image similar target detection device provided by the embodiment of the present application, as shown in Figure 3 The visual prompt guided remote sensing image similar target detection device provided by the present application is described below. The visual prompt guided remote sensing image similar target detection device described below can be correspondingly referred to the visual prompt guided remote sensing image similar target detection method described above.

[0091] The data acquisition and preprocessing module 31 is used for remote sensing data selection and preprocessing to obtain preprocessed image data and preprocessed visual prompt data.

[0092] The model construction module 32 is configured to construct a target detection model; the target detection model is composed of a CNN backbone network, a Transformer and a loss function; the Transformer includes an encoder, a decoder and a feedforward neural network (FNN);

[0093] The data fusion and output module 33 is configured to input the feature extraction of the preprocessed image data by the CNN backbone network into the encoder of the Transformer; add the preprocessed visual cue data after the feature extraction by the CNN backbone network to the query object and input into the decoder of the Transformer; fuse the preprocessed image and the visual cue processing data in the decoder, and then output the bounding box and the label by the FNN to obtain the prediction result.

[0094] The final model acquisition module 34 is configured to train the network by using the loss function and the prediction result to obtain a final target detection model.

[0095] Figure 4 An example of an entity structure diagram of an electronic device is shown in Figure 4 As shown, the electronic device can include a processor 410, a communications interface 420, a memory 430 and a communications bus 440, wherein the processor 410, the communications interface 420 and the memory 430 complete mutual communication through the communications bus 440. The processor 410 can invoke the logical instructions in the memory 430 to execute the visual cue guided remote sensing image similar target detection method.

[0096] In addition, the logical instructions in the memory 430 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk and various program code storage media.

[0097] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the remote sensing image similar target detection method guided by visual cues provided by the above methods.

[0098] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the remote sensing image similar target detection method guided by visual cues provided by the above methods.

[0099] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0100] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for detecting similar targets in remote sensing images guided by visual cues, characterized in that: The following steps are involved: Remote sensing data selection and preprocessing to obtain preprocessed image data and preprocessed visual cue data, including: labeling the remote sensing data to obtain the category, width, height, and center point coordinate information of each target in the image, and organizing them into a dictionary; classifying the categories of targets in the remote sensing data into visible categories and invisible categories, screening and retaining the visible categories, and obtaining retained remote sensing data; randomly selecting a target from the retained remote sensing data as a visual cue, and setting its label to the same category as the visual cue; regularizing and randomly resizing the image in the retained remote sensing data; randomly adjusting the brightness, contrast, saturation, and hue of the visual cue, and then converting it into a grayscale image, and then performing Gaussian blurring and regularization; Constructing a target detection model; the target detection model consists of a CNN backbone network, a Transformer, and a loss function; the Transformer includes an encoder, a decoder, and a feedforward neural network FNN; The CNN backbone network is used to extract features from preprocessed image data and input them into the encoder of the Transformer; the CNN backbone network is used to extract features from preprocessed visual cue data and then add them to the query object and input them into the decoder of the Transformer; the preprocessed image and visual cue processing data are fused in the decoder, and then output bounding boxes and labels through the FNN to obtain prediction results; the method for fusing the preprocessed image and visual cue processing data in the decoder is as follows: the preprocessed image and visual cue processing data are fused in the decoder; the preprocessed visual cue data is passed through the CNN backbone network and the pooling layer, and then added to the query object to obtain a first data set, and then passed through the multi-head attention mechanism layer and the normalization layer to output the attention vector Query; the preprocessed image data is passed through the CNN backbone network and the encoder to output a set of attention vectors Key and Value; the Query, Key and Value are input into the multi-head attention mechanism layer and the normalization layer to achieve data fusion; The network is trained using the loss function and prediction results to obtain the final target detection model.

2. The method for detecting similar targets in remote sensing images using visual cues according to claim 1, wherein: The target detection model consists of a main network, a branch network and a loss function; Among them, the main network is composed of CNN backbone network, encoder, decoder and FNN connection in sequence; The branch network is composed of a CNN backbone network and a pooling layer in sequence; the branch network is connected to the decoder of the main network; The CNN backbone network of the main network and the CNN backbone network of the branch network are the same network and share parameters.

3. The method for detecting similar targets in remote sensing images using visual cues according to claim 1, wherein: The expression of the loss function is: Among them, y represents the true value; represents the predicted value; Indicates the binary classification result. If it is 0, it means the category is the same as the visual prompt, and if it is 1, it means it is different. Represents the prediction box result; Predict matching order; Indicates the i target category truth values, i Indicates the image i goals; Indicates the i The true value of the target position; Indicates the i The target category true value corresponds to the weight coefficient; represents the cross entropy loss; represents the union of L1 loss and GIoU loss, Indicates judgment conditions Is it true? 4. The method for detecting similar targets in remote sensing images using visual cues according to claim 1, wherein: The training method is: The prediction results are combined with the true value of the bounding box and the label to calculate the loss function, and the gradient of the target detection model parameters is calculated, and the weights of the target detection model are updated according to the gradient.

5. A device for detecting similar targets in remote sensing images guided by visual cues, characterized in that: The remote sensing image similar target detection device is used to implement the remote sensing image similar target detection method guided by visual cues according to any one of claims 1 to 4, comprising: Data acquisition and preprocessing module, used for remote sensing data selection and preprocessing, obtaining preprocessed image data and preprocessed visual cue data; A model building module is used to build a target detection model; the target detection model consists of a CNN backbone network, a Transformer, and a loss function; the Transformer includes an encoder, a decoder, and a feedforward neural network FNN; The data fusion and output module is used to extract features from preprocessed image data using the CNN backbone network and input them into the Transformer encoder. The CNN backbone network is used to extract features from preprocessed visual cue data, which is then added to the query object and input into the Transformer decoder. The preprocessed image and visual cue data are fused in the decoder and then output by the FNN with bounding boxes and labels to obtain the prediction results. The final model acquisition module is used to train the network using the loss function and prediction results to obtain the final target detection model.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for detecting similar targets in remote sensing images guided by visual cues as described in any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting similar targets in remote sensing images guided by visual cues as claimed in any one of claims 1 to 4 is implemented.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for detecting similar targets in remote sensing images guided by visual cues as claimed in any one of claims 1 to 4 is implemented.

Images