Remote sensing image text retrieval method based on global-local information interactive attention

Abstract

This invention discloses a remote sensing image text retrieval method that uses global-local information interaction to pay attention to the text. The method comprises the following steps: 1. obtaining regional feature representations of remote sensing images through a feature extraction network based on the Faster R-CNN model; 2. obtaining word-level feature representations of text with contextual information through a feature extraction network based on the BERT model; 3. constructing a global-local attention module based on an attention mechanism to obtain remote sensing image features that contain global-local information; 4. obtaining overall features of the remote sensing image and text through a feature aggregation network; and 5. training an optimal retrieval model by constraining the feature distance between positive and negative samples using a bidirectional triple ranking loss function. By obtaining remote sensing image features that use global-local information interaction to pay attention to the text and text features that contain contextual information, the method significantly improves the accuracy of remote sensing image text retrieval.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and in particular to a remote sensing image text retrieval method based on global-local information interactive attention. Background Art

[0002] With the continuous development of satellite and remote sensing technology, remote sensing satellites generate a vast amount of remote sensing imagery daily. Retrieving valuable data from massive remote sensing image databases based on semantics is crucial for remote sensing data mining and application. Remote sensing image text retrieval aims to enable cross-referencing between remote sensing images and text by associating them. Efficient remote sensing image text retrieval can meet the growing demand for remote sensing image data retrieval and mining, and improve the accuracy of remote sensing images in applications such as remote sensing image analysis and smart agriculture.

[0003] The goal of cross-modal remote sensing image-text retrieval is to retrieve matching text for a given remote sensing image, or to retrieve the corresponding remote sensing image based on a given text. The main difficulty in cross-modal remote sensing image text retrieval lies in the fact that features from different modalities reside in different feature spaces and are heterogeneous in underlying data structures but related in top-level semantics. For example, text encoding features and remote sensing image encoding features representing the same topic reside in completely different feature spaces. Although they both represent the same topic, their feature vectors are completely different.

[0004] However, existing methods primarily focus on the global features of remote sensing images, ignoring local features that can reflect salient objects, resulting in reduced retrieval accuracy. Furthermore, most existing algorithms simply fuse features from remote sensing images or text, failing to fully utilize the semantic information of the features, resulting in poor retrieval results. Summary of the Invention

[0005] In order to overcome the problems existing in the prior art, the present invention proposes a remote sensing image text retrieval method based on global-local information interaction attention, in order to fully learn the visual semantic information of the interdependence between remote sensing image objects, as well as the aggregation of remote sensing image region-level features and text word-level features at the semantic level, thereby improving the accuracy of remote sensing image text retrieval.

[0006] In order to achieve the above-mentioned object, the present invention adopts the following technical solutions:

[0007] The present invention provides a remote sensing image text retrieval method with global-local information interactive attention, which comprises the following steps:

[0008] Step 1: Build an image region feature extraction network based on the Faster R-CNN model, including: remote sensing image object box detection module and remote sensing image region-level feature extraction module;

[0009] Step 1.1: After obtaining a remote sensing image dataset and its corresponding text dataset, the remote sensing image object box detection module uses the Faster R-CNN model to process any remote sensing image pic in the remote sensing image dataset to obtain q object boxes with the highest confidence scores.

[0010] Step 1.2: Use the BUTD model to extract features from q object frames of the remote sensing image pic, and obtain a set of regional features P = {p1, ..., p i ,…,p q}, p i Represents the regional features corresponding to the i-th object frame of the remote sensing image pic, D v represents the dimension of the feature, q represents the total number of regional features; i = 1,…,q;

[0011] Step 2: Build a text word-level feature extraction network based on the BERT model and use it to process the text corresponding to the remote sensing image pic to obtain a set of word-level features E;

[0012] Step 3: Build a global-local attention processing module and use it to process P to obtain a set of information interaction attention feature vectors F of the remote sensing image pic;

[0013] Step 4: Build a spatial attention aggregation module and use it to process F and E to obtain the feature v of the remote sensing image pic and the feature t of its corresponding text;

[0014] Step 5: Construct a remote sensing image text retrieval network consisting of a remote sensing image region-level feature extraction network, a text word-level feature extraction network, a global-local attention processing module, and a spatial attention aggregation module. Establish a loss function L for training the network to obtain the optimal remote sensing image text retrieval network for retrieving remote sensing images and texts:

[0015] Step 5.1: Use formula (1) to construct the network loss function L;

[0016]

[0017] In formula (1), (v, t) represents any set of matched remote sensing image-text feature pairs; δ is a predefined margin parameter, s(·) is the cosine distance function; (v -,t) represents any set of text features t and its remote sensing image feature v- that does not match the remote sensing image feature, (v,t - ) represents any set of remote sensing image features v and their unmatched text features t-remote sensing image-text feature pairs;

[0018] Step 5.2: Input the remote sensing image dataset and its corresponding text dataset into the image-text retrieval network, and continuously optimize the loss function L through the Adam optimizer to adjust the network parameters, thereby obtaining the optimal remote sensing image text retrieval network.

[0019] The remote sensing image text retrieval method with global-local information interactive attention according to the present invention is also characterized in that the text word-level feature extraction network in step 2 includes: a subword segmentation module and a text word-level feature extraction module;

[0020] Step 2.1, the text subword segmenter module uses the WordPiece segmenter to split the text corresponding to the remote sensing image pic into subword sequences;

[0021] Step 2.2: The text word-level feature extraction module sequentially inputs the split subword sequence into the pre-trained BERT model for processing, and obtains a set of word-level features E = {e1, ..., e j ,…,e s}, e j represents the features of the j-th word, D t represents the dimension of the feature, s represents the total number of words in the text, j = 1,…,s.

[0022] The global-local attention processing module in step 3 includes: a global attention feature generation module, a similarity matrix construction module, a local calibration feature generation module, and a global-local attention feature fusion module;

[0023] Step 3.1: The global attention feature generation module projects a set of regional features P into the hidden space through three different linear layers, and obtains the global attention matrix M through formula (2), and then generates the normalized global attention map A through formula (3): g , the global attention feature P is obtained through formula (4) g ;

[0024] M=f q (P)·f k (P) T (2)

[0025]

[0026] P g =Ag ·f v (P) (4)

[0027] In formulas (2) to (4), fq(·), fk(·), and fv(·) represent three different linear mapping layers, d h represents the dimension of the latent space, T represents transposition, and Softmax represents the activation function;

[0028] Step 3.2: The similarity matrix construction module constructs a similarity matrix with a mask window through formula (5):

[0029]

[0030] In formula (5), w is the size of the mask window, M ij represents the element in row i and column j in M, express The element in the i-th row and j-th column, i,j∈[0,q-1];

[0031] Step 3.3: The local calibration feature generation module generates a similarity matrix with a mask window. Generate the local calibration attention map A through formula (6) l , and then use formula (7) to get the local calibration feature P l ;

[0032]

[0033] P l =A l ·f v (P) (7)

[0034] Step 3.4: The global-local attention feature fusion module obtains a set of information interaction attention feature vectors F = {f1,…,f i ,…,f q}, Represents the characteristics of the i-th information interaction attention:

[0035] P o =α·P g +(1-α)·P l (8)

[0036] F=LN(X+f h (Norm(P o ))) (9)

[0037] In formula (8)-formula (9), α represents the weight to be learned, P orepresents the fused features, Norm(·) represents the combination of power normalization and L2 normalization, and fh(·) transforms the features from the dimension d of the latent space h Mapping back to the dimension D of the regional features v , LN represents layer normalization, and + represents residual connection.

[0038] The spatial attention aggregation module in step 4 includes: an image region-level feature aggregation module and a text word-level feature aggregation module;

[0039] Step 4.1: The image region-level feature aggregation module obtains the projection feature vector V = {v1,…,v i ,…,v q}, vq∈Rd represents the i-th projection feature;

[0040] v i =W v ·f i +b v (10)

[0041] In formula (10), is the projection matrix of the remote sensing image pic, b v is the bias variable of the remote sensing image pic;

[0042] Step 4.2: The image region-level feature aggregation module obtains the weight matrix M of the remote sensing image pic through formula (11): v ∈R q×d ; Then, the feature v∈R of the remote sensing image pic is obtained by formula (12): d ;

[0043] M v =σ(V·M1) (11)

[0044]

[0045] In formula (11)-formula (12), M1∈R d×d is the parameter matrix to be learned of the remote sensing image pic, σ represents the softmax normalization in the column direction, and ⊙ represents the dot product operation;

[0046] Step 4.3: The text word-level feature aggregation module obtains the projection feature vector T of the remote sensing image pic corresponding to the text through formula (13) j ,…,t s}, tj∈Rd represents the j-th projection feature;

[0047] t j =W t ·e j +bt (13)

[0048] In formula (13), is the projection matrix of the remote sensing image pic corresponding to the text, b t is the bias variable of the remote sensing image pic corresponding to the text;

[0049] Step 4.4: The text word-level feature aggregation module obtains the weight matrix M of the remote sensing image pic corresponding to the text through formula (14): v ∈R s×d ; Then, we use formula (15) to get the feature t∈R of the remote sensing image pic corresponding to the text d ;

[0050] M t =σ(T·M2) (14)

[0051]

[0052] In formula (14)-(15), M2∈R d×d is the parameter matrix to be learned corresponding to the text of the remote sensing image pic, σ represents the softmax normalization in the column direction, and ⊙ represents the dot multiplication operation.

[0053] The electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the remote sensing image text retrieval method, and the processor is configured to execute the program stored in the memory.

[0054] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium. The characteristic of the computer program is that when the computer program is run by a processor, the steps of the remote sensing image text retrieval method are executed.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] 1. This paper proposes a global-local attention processing module. By modeling the relationships between image regions, this module generates new features that focus more on salient areas in remote sensing images. It then uses a similarity matrix and adaptive fusion to effectively capture detailed changes and enhance local features. This module achieves good encoding performance with fewer parameters and computational cost.

[0057] 2. This paper proposes a spatial attention aggregation module, which realizes the interaction of semantic dimensions in a shared space. At the semantic level, by combining element-by-element attention with aggregation, it considers the semantic consistency between the same dimension and the semantic difference between different dimensions, thereby obtaining finer-grained and more accurate remote sensing image features and text features.

[0058] 3. This paper uses the BERT model, which can better describe word meanings, to extract word features from text. The model focuses on exploring the relationship between words, can integrate left and right context information and generate context-aware word representations, thereby improving the accuracy of remote sensing image text retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a structural diagram of the remote sensing image text retrieval network with global-local information interaction attention in the present invention;

[0060] Figure 2 This is a diagram of the global-local attention processing module in the present invention;

[0061] Figure 3 This is a diagram of the spatial attention aggregation module in the present invention. DETAILED DESCRIPTION

[0062] The present invention will be further described below in conjunction with the accompanying drawings:

[0063] In this embodiment, a remote sensing image text retrieval method based on global-local information interactive attention is proposed. Figure 1 As shown, the method includes the following steps:

[0064] Step 1: Build an image region feature extraction network based on the Fast R-CNN model, including: remote sensing image object box detection module and remote sensing image region-level feature extraction module;

[0065] Step 1.1: After obtaining the remote sensing image dataset and its corresponding text dataset, the remote sensing image object box detection module uses the Fast R-CNN model to process any remote sensing image pic in the remote sensing image dataset and obtain the q object boxes with the highest confidence scores;

[0066] Step 1.2: Use the BUTD model to extract features from q object frames of the remote sensing image pic, and obtain a set of regional features P = {p1, ..., p i ,…,p q}, p i Represents the regional features corresponding to the i-th object frame of the remote sensing image pic, D v represents the dimension of the feature, q represents the total number of regional features; i = 1,…,q.

[0067] In this example, feature extraction was performed on remote sensing images from two commonly used remote sensing image text datasets: RSICD and RSITMD. RSICD, which contains 10,921 remote sensing images, each annotated with five sentences, is the largest remote sensing image text dataset. 80% of the dataset was used as the training set, 10% as the validation set, and 10% as the test set. RSITMD, another remote sensing image text dataset, contains 4,743 remote sensing images, each annotated with five sentence-level descriptions. 80% of the dataset was used as the training set, 10% as the validation set, and 10% as the test set.

[0068] Step 2: Build a text word-level feature extraction network based on the BERT model and use it to process the text corresponding to the remote sensing image pic to obtain a set of word-level features E, including: a subword segmenter module and a text word-level feature extraction module;

[0069] Step 2.1, the text subword segmenter module uses the WordPiece segmenter to split the text corresponding to the remote sensing image pic into subword sequences;

[0070] Step 2.2: The text word-level feature extraction module sequentially inputs the split subword sequence into the pre-trained BERT model for processing, and obtains a set of word-level features E = {e1,…,e j ,…,e s}, e j represents the features of the j-th word, D t represents the dimension of the feature, s represents the total number of words in the text, j = 1,…,s.

[0071] Step 3: Build a global-local attention processing module based on the attention mechanism and use it to process P to obtain a set of information interaction attention feature vectors F of the remote sensing image pic, including: global attention feature generation module, similarity matrix construction module, local calibration feature generation module, and global-local attention feature fusion module;

[0072] like Figure 2 As shown in the figure, the global-local attention processing module based on the attention mechanism can model the relationship between the various regions of the remote sensing image, and the new features obtained will focus more on the salient areas in the remote sensing image.

[0073] Step 3.1: The global attention feature generation module projects a set of regional features P into the hidden space through three different linear layers, and obtains the global attention matrix M through formula (2). Then, it applies Softmax through formula (3) to generate the normalized global attention map A. g , the projection feature is re-weighted through the global attention map through formula (4) to obtain the global attention feature Pg ;

[0074] M=f q (P)·f k (P) T (2)

[0075]

[0076] P g =A g ·f v (P) (4)

[0077] In formulas (2) to (4), fq(·), fk(·), and fv(·) represent three different linear mapping layers, d h represents the dimension of the latent space, T represents transpose, and Softmax represents the activation function.

[0078] Step 3.2: The similarity matrix construction module constructs a similarity matrix with a mask window through formula (5)

[0079]

[0080] In formula (5), w is the size of the mask window, M ij represents the element in row i and column j in M, express The element in the i-th row and j-th column, i,j∈[0,q-1].

[0081] Step 3.3: The local calibration feature generation module generates similarity matrices with mask windows. The Softmax normalization is also applied through formula (6) to generate the local calibration attention map A l Then, the projection feature is re-weighted by the local calibration attention map through formula (7) to obtain the local calibration feature P l ;

[0082]

[0083] P l =A l ·f v (P) (7)

[0084] Step 3.4: The global-local attention feature fusion module obtains a set of information interaction attention feature vectors F = {f1,…,f i ,…,f q}, Represents the characteristics of the i-th information interaction attention:

[0085] P o =α·P g +(1-α)·P l (8)

[0086] F=LN(X+f h (Norm(P o ))) (9)

[0087] In formula (8)-formula (9), α represents the weight to be learned, P o represents the fused features, Norm(·) represents the combination of power normalization and L2 normalization, and fh(·) transforms the features from the dimension d of the latent space h Mapping back to the dimension D of the regional features v , LN represents layer normalization, and + represents residual connection; this module can effectively capture detail changes by using similarity matrix and adaptive fusion, enhance local features, and thus obtain better feature representation.

[0088] Step 4: Build a spatial attention aggregation module and use it to process F and E to obtain the feature v of the remote sensing image pic and the feature t of its corresponding text, including: a remote sensing image region-level feature aggregation module and a text word-level feature aggregation module;

[0089] Spatial attention aggregation module, such as Figure 3 As shown in the figure, this module realizes the interaction of semantic dimensions in the shared space. At the semantic level, by combining element-by-element attention with aggregation, it considers the semantic consistency between the same dimension and the semantic difference between different dimensions, thereby obtaining more fine-grained and accurate remote sensing image features and text features.

[0090] Step 4.1: The image region-level feature aggregation module converts the fully connected layer into the d-dimensional shared space through formula (10) to obtain the projection feature vector V = {v1,…,v i ,…,v q}, vq∈Rd represents the i-th projection feature;

[0091] v i =W v ·f i +b v (10)

[0092] In formula (10), is the projection matrix of the remote sensing image pic, b v is the bias variable of the remote sensing image pic;

[0093] Step 4.2: The image region-level feature aggregation module obtains the weight matrix M of the remote sensing image pic through formula (11): v ∈Rq×d ; Then, the regional level features of the remote sensing image are merged through formula (12) to obtain the feature v∈R of the remote sensing image pic d ;

[0094] M v =σ(V·M1) (11)

[0095]

[0096] In formula (11)-formula (12), M1∈R d×d is the parameter matrix to be learned of the remote sensing image pic, σ represents the softmax normalization in the column direction, and ⊙ represents the dot multiplication operation.

[0097] Step 4.3: The text word-level feature aggregation module converts the fully connected layer into the d-dimensional shared space through formula (13) to obtain the projection feature vector T = {t1,…,t j ,…,t s}, tj∈Rd represents the j-th projection feature;

[0098] t j =W t ·e j +b t (13)

[0099] In formula (13), is the projection matrix of the remote sensing image pic corresponding to the text, b t is the bias variable of the remote sensing image pic corresponding to the text;

[0100] Step 4.4: The text word-level feature aggregation module obtains the weight matrix M of the remote sensing image pic corresponding to the text through formula (14): v ∈R s×d ; Then, the word-level features are combined through formula (15) to obtain the feature t∈R of the remote sensing image pic corresponding to the text d ;

[0101] M t =σ(T·M2) (14)

[0102]

[0103] In formula (14)-(15), M2∈R d×d is the parameter matrix to be learned corresponding to the text of the remote sensing image pic, σ represents the softmax normalization in the column direction, and ⊙ represents the dot multiplication operation.

[0104] Step 5: Construct a remote sensing image text retrieval network consisting of a remote sensing image region-level feature extraction network, a text word-level feature extraction network, a global-local attention processing module, and a spatial attention aggregation module. Establish a loss function L for training the network to obtain the optimal remote sensing image text retrieval network for retrieving remote sensing images and texts:

[0105] Step 5.1: Use formula (1) to construct the network loss function L;

[0106]

[0107] In formula (1), (v, t) represents any set of matched remote sensing image-text feature pairs; δ is a predefined margin parameter, s(·) is the cosine distance function; (v - ,t) represents any set of text features t and its unmatched remote sensing image features v-remote sensing image-text feature pairs, (v,t - ) represents any set of remote sensing image features v and their unmatched text features t-remote sensing image-text feature pairs;

[0108] Step 5.2: Input the remote sensing image dataset and its corresponding text dataset into the image-text retrieval network, and continuously optimize the loss function L through the Adam optimizer to adjust the network parameters to obtain the optimal remote sensing image-text retrieval network.

[0109] In this embodiment, an electronic device includes a memory and a processor. The memory is used to store a program that supports the processor to perform the above-mentioned remote sensing image text retrieval method. The processor is configured to execute the program stored in the memory.

[0110] In this embodiment, a computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the remote sensing image text retrieval method are executed.

Claims

1. A remote sensing image text retrieval method with global-local information interactive attention, characterized by: The steps include: Step 1: Build an image region feature extraction network based on the FasterR-CNN model, including: remote sensing image object box detection module and remote sensing image region-level feature extraction module; Step 1.1: After obtaining a remote sensing image dataset and its corresponding text dataset, the remote sensing image object box detection module uses the Faster R-CNN model to process any remote sensing image pic in the remote sensing image dataset to obtain q object boxes with the highest confidence scores. Step 1.2: Use the BUTD model to extract features from q object frames of the remote sensing image pic, and obtain a set of regional features P = {p1, ..., p i ,…,p q }, p i Represents the regional features corresponding to the i-th object frame of the remote sensing image pic, D v represents the dimension of the feature, q represents the total number of regional features; i = 1,…,q; Step 2: Build a text word-level feature extraction network based on the BERT model and use it to process the text corresponding to the remote sensing image pic to obtain a set of word-level features E; Step 3: Build a global-local attention processing module and use it to process P to obtain a set of information interaction attention feature vectors F of the remote sensing image pic; Step 4: Build a spatial attention aggregation module and use it to process F and E to obtain the feature v of the remote sensing image pic and the feature t of its corresponding text; Step 5: Construct a remote sensing image text retrieval network consisting of a remote sensing image region-level feature extraction network, a text word-level feature extraction network, a global-local attention processing module, and a spatial attention aggregation module. Establish a loss function L for training the network to obtain the optimal remote sensing image text retrieval network for retrieving remote sensing images and texts: Step 5.1: Use formula (1) to construct the network loss function L; In formula (1), (v, t) represents any set of matched remote sensing image-text feature pairs; δ is a predefined margin parameter, s(·) is the cosine distance function; (v - ,t) represents any set of text features t and its remote sensing image feature v- that does not match the remote sensing image feature, (v,t - ) represents any set of remote sensing image features v and their unmatched text features t- remote sensing image-text feature pairs; Step 5.2: Input the remote sensing image dataset and its corresponding text dataset into the image-text retrieval network, and continuously optimize the loss function L through the Adam optimizer to adjust the network parameters, thereby obtaining the optimal remote sensing image text retrieval network.

2. The remote sensing image text retrieval method based on global-local information interactive attention according to claim 1 is characterized in that: The text word-level feature extraction network in step 2 includes: a subword segmentation module and a text word-level feature extraction module; Step 2.1, the text subword segmenter module uses the WordPiece segmenter to split the text corresponding to the remote sensing image pic into subword sequences; Step 2.2: The text word-level feature extraction module sequentially inputs the split subword sequence into the pre-trained BERT model for processing, and obtains a set of word-level features E = {e1, ..., e j ,…,e s }, e j represents the features of the j-th word, D t represents the dimension of the feature, s represents the total number of words in the text, j = 1,…,s.

3. The remote sensing image text retrieval method based on global-local information interactive attention according to claim 2 is characterized in that: The global-local attention processing module in step 3 includes: a global attention feature generation module, a similarity matrix construction module, a local calibration feature generation module, and a global-local attention feature fusion module; Step 3.1: The global attention feature generation module projects a set of regional features P into the hidden space through three different linear layers, and obtains the global attention matrix M through formula (2), and then generates the normalized global attention map A through formula (3): g , the global attention feature P is obtained through formula (4) g ; M=f q (P)·f k (P) T (2) P g =A g ·f v (P) (4) In formulas (2) to (4), fq(·), fk(·), and fv(·) represent three different linear mapping layers, d h represents the dimension of the latent space, T represents transposition, and Softmax represents the activation function; Step 3.2: The similarity matrix construction module constructs a similarity matrix with a mask window through formula (5): In formula (5), w is the size of the mask window, M ij represents the element in row i and column j in M, express The element in the i-th row and j-th column, i,j∈[0,q-1]; Step 3.3: The local calibration feature generation module generates a similarity matrix with a mask window. Generate the local calibration attention map A through formula (6) l , and then use formula (7) to get the local calibration feature P l ; P l =A l ·f v (P) (7) Step 3.4: The global-local attention feature fusion module obtains a set of information interaction attention feature vectors F = {f1,…,f i ,…,f q }, Represents the characteristics of the i-th information interaction attention: P o =α·P g +(1-α)·P l (8) F=LN(X+f h (Norm(P o ))) (9) In formula (8)-formula (9), α represents the weight to be learned, P o represents the fused features, Norm(·) represents the combination of power normalization and L2 normalization, and fh(·) transforms the features from the dimension d of the latent space h Mapping back to the dimension D of the regional features v , LN represents layer normalization, and + represents residual connection.

4. The remote sensing image text retrieval method based on global-local information interactive attention according to claim 3 is characterized in that: The spatial attention aggregation module in step 4 includes: an image region-level feature aggregation module and a text word-level feature aggregation module; Step 4.1: The image region-level feature aggregation module obtains the projection feature vector V = {v1,…,v i ,…,v q }, vq∈Rd represents the i-th projection feature; v i =W v ·f i +b v (10) In formula (10), is the projection matrix of the remote sensing image pic, b v is the bias variable of the remote sensing image pic; Step 4.2: The image region-level feature aggregation module obtains the weight matrix M of the remote sensing image pic through formula (11): v ∈R q×d ; Then, the feature v∈R of the remote sensing image pic is obtained by formula (12): d ; M v =σ(V·M1) (11) In formula (11)-formula (12), M1∈R d×d is the parameter matrix to be learned of the remote sensing image pic, σ represents the softmax normalization in the column direction, and ⊙ represents the dot product operation; Step 4.3: The text word-level feature aggregation module obtains the projection feature vector T of the remote sensing image pic corresponding to the text through formula (13) j ,…,t s }, tj∈Rd represents the j-th projection feature; t j =W t ·e j +b t (13) In formula (13), is the projection matrix of the remote sensing image pic corresponding to the text, b t is the bias variable of the remote sensing image pic corresponding to the text; Step 4.4: The text word-level feature aggregation module obtains the weight matrix M of the remote sensing image pic corresponding to the text through formula (14): v ∈R s×d ; Then, we use formula (15) to get the feature t∈R of the remote sensing image pic corresponding to the text d ; M t =σ(T·M2) (14) In formula (14)-(15), M2∈R d×d is the parameter matrix to be learned corresponding to the text of the remote sensing image pic, σ represents the softmax normalization in the column direction, and ⊙ represents the dot multiplication operation.

5. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the remote sensing image text retrieval method according to any one of claims 1 to 4, and the processor is configured to execute the program stored in the memory.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the remote sensing image text retrieval method according to any one of claims 1 to 4 are executed.

Images