A step-by-step detection network for remote sensing images based on Transformer region proposal

Abstract

The present invention discloses a remote sensing image step-by-step detection network based on Transformer region proposal. The network includes: a first input layer, a label embedding layer, a feature fusion layer, a Transformer encoder, a Transformer decoder, and a fully connected layer; a second input layer, a variational autoencoder, an embedding layer, and a single attention layer; the first input layer is connected to the label embedding layer, the label embedding layer is connected to the feature fusion layer, and the feature fusion layer is connected to the Transformer encoder. The present invention solves the technical problems that the existing remote sensing image target detection technology processes large-size images by reducing the input remote sensing image, or crops the remote sensing image into smaller image slices and sets the resampling rate to perform target detection in the remote sensing image, resulting in long detection time, high resource utilization, loss of small target information, and a high missed detection rate.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a remote sensing image step-by-step detection network based on Transformer region proposal. Background Art

[0002] In recent years, advances in deep learning technology have driven significant progress in object detection tasks. Single-stage detection networks, such as the YOLO series, utilize convolutional neural networks (CNNs) to directly extract features from the entire image and then apply a distributed grid to the detection head to predict object information within that grid. This approach significantly improves detection speed and achieves comparable accuracy to other detection models without relying on candidate region proposal mechanisms, playing a significant role in the development of object detection in industrial applications. However, as model parameters increase, CNN-based methods experience limited improvements in learning capabilities. To address this, an attention mechanism can be introduced to associate global features, allowing the model to learn more feature subspace information, improving its representational capabilities and decision accuracy.

[0003] In recent years, the Transformer has been successfully introduced from the field of natural language processing (NLP) into computer vision, achieving remarkable results. Unlike CNNs, the Transformer associates global information in the image through a self-attention mechanism. In object detection, models such as DETR map the feature maps extracted by CNN into an embedding sequence, which is fed into the Transformer encoding and decoding module, treating object detection as a query process. After the query and image features are mutually attentive through the decoder, the FPN of the detection head outputs the location and category information of the target. The improved Deformable-DETR uses a deformable multi-head attention mechanism, allowing the model to focus on specific salient areas in the image rather than the entire image, thereby improving convergence speed and processing efficiency, and significantly enhancing the ability to detect small objects in high-resolution images.

[0004] The above models have achieved good results in the field of target detection. However, existing target detection algorithms mainly target common images, while there is little research on remote sensing images with large pixel sizes and small target proportions. Most methods crop the original high-resolution image into smaller image slices and set a resampling rate to prevent the target from being cut by the cropping line, and then train and detect the model. However, this method will lead to a geometric increase in detection time and resources. Other methods process large-size images by reducing the input image, which will lead to the loss of small target information and increase the missed detection rate. Although these strategies are simple and direct, they can lead to serious detection errors or increase computational costs, which is not conducive to meeting the high real-time and safety requirements in fields such as aviation, aerospace, and autonomous driving.

[0005] To solve the above problems, the present invention designs a feature learning-based Transformer region proposal step-by-step detection network to shorten the detection time of large-scale remote sensing images as much as possible while ensuring detection accuracy. Summary of the Invention

[0006] An embodiment of the present invention provides a remote sensing image step-by-step detection network based on Transformer region proposal, which at least solves the technical problems of existing remote sensing image target detection technology that processes large-size images by reducing the input remote sensing image, or crops the remote sensing image into smaller image slices and sets the resampling rate to perform target detection in the remote sensing image, resulting in long detection time, high resource utilization, loss of small target information and high missed detection rate.

[0007] According to one aspect of an embodiment of the present invention, a remote sensing image step-by-step detection network based on Transformer region proposal is provided. The network may include: a first input layer, a label embedding layer, a feature fusion layer, a Transformer encoder, a Transformer decoder, and a fully connected layer; a second input layer, a variational autoencoder, an embedding layer, and a single attention layer; the first input layer is connected to the label embedding layer, the label embedding layer is connected to the feature fusion layer, and the feature fusion layer is connected to the Transformer encoder; the second input layer is connected to the variational autoencoder, the variational autoencoder is connected to the embedding layer, the embedding layer is connected to the feature fusion layer, and the feature fusion layer is connected to the single attention layer; the single attention layer and the Transformer encoder are connected to the Transformer decoder, and the Transformer decoder is connected to the fully connected layer; the first input layer obtains the original remote sensing image, inputs the original remote sensing image into the label embedding layer, and generates a memory latent variable; the feature fusion layer combines the memory latent variable with the position encoding Add the input to the Transformer encoder to generate the target latent variable; the second input layer obtains the feature map of the large-scale target distribution corresponding to the original remote sensing image and inputs the feature map of the large-scale target distribution into the variational autoencoder to obtain the feature map of the Gaussian distribution; the feature map of the Gaussian distribution is input into the embedding layer to obtain a high-dimensional feature map, and the feature fusion layer adds the high-dimensional feature map to the position code to obtain the prior detection feature; after the prior detection feature is input into the single attention layer, a target vector is generated, wherein the prior detection feature includes: an initial query vector, an initial key vector, and an initial value vector; the target vector generated by the single attention layer and the target latent variable generated by the Transformer encoder are input into the Transformer decoder to obtain the shallow features of the original remote sensing image, and the shallow features of the original remote sensing image are input into the fully connected layer to obtain a two-dimensional distribution probability map of small-scale targets in the original remote sensing image.

[0008] Optionally, the expression of the prior detection feature is:

[0009]

[0010] in, is the initial query vector, is the initial key vector, is the initial value vector, is the feature map of the large-scale target distribution corresponding to the original remote sensing image, is a Gaussian distribution, is the position code, and its specific expression is , i is its sequence length, n is the feature dimension, k is a positive integer, is the convolution operation, It is the feature map of Gaussian distribution corresponding to the feature map of large-scale target distribution.

[0011] Optionally, after inputting the prior detection features into the single attention layer, the expression for generating the target vector is:

[0012]

[0013] in, is the target query vector, is the query vector parameter, is the target key vector, key vector parameters, is the target value vector, is the value vector parameter;

[0014]

[0015] in, is the transpose, is the dimension of the K vector, is the target vector.

[0016] Alternatively, the loss function of a remote sensing image step-by-step detection network based on Transformer region proposal is expressed as:

[0017]

[0018] in, is the loss function value, for and The cross entropy loss function value is, It is the small-size target two-dimensional distribution feature map of the original remote sensing image output by the step-by-step detection network. is the real two-dimensional distribution feature map of small-size targets in the original remote sensing image, KL is the regularization function, is random noise, The two-dimensional distribution feature map of small-sized targets in the original remote sensing image output by the step-by-step detection network corresponds to a normal distribution with a mean of , the variance is , is a standard normal distribution with a mean of 0 and a variance of 1.

[0019] Optionally, an evaluation index of a remote sensing image step-by-step detection network based on Transformer region proposal is the average slice sequence overlap, and the process of determining the average slice sequence overlap is: obtaining the number of targets contained in each slice of the original remote sensing image, the probability value of each slice of the small-size target two-dimensional distribution feature map of the original remote sensing image, the initial number of slices and the total number of slices; sorting the number of targets contained in each slice from large to small to obtain a first sequence; sorting the probability value of each slice from large to small to obtain a second sequence; based on the initial number of slices, the first sequence corresponding to the initial number of slices and the second sequence corresponding to the initial number of slices, obtaining the overlap between the first sequence corresponding to the initial number of slices and the second sequence; based on the overlap between the first sequence corresponding to the initial number of slices and the second sequence and the total number of slices, obtaining the evaluation index of the second sequence.

[0020] Optionally, the expression for the degree of overlap between the first sequence corresponding to the initial number of slices and the second sequence corresponding to the initial number of slices is obtained based on the initial number of slices, the first sequence corresponding to the initial number of slices, and the second sequence corresponding to the initial number of slices:

[0021]

[0022] in, is the overlap between the first and second sequences corresponding to the initial number of slices, is the initial number of slices, is the first sequence corresponding to the initial number of slices, is the second sequence corresponding to the initial number of slices.

[0023] Optionally, the expression for obtaining the evaluation index of the second sequence based on the overlap between the first sequence and the second sequence corresponding to the initial number of slices and the total number of slices is:

[0024]

[0025] is the evaluation index of the second sequence, is the total number of slices.

[0026] Beneficial effects of the present invention:

[0027] (1) The Transformer model used can capture global information in the image through the self-attention mechanism. This feature enables the model to fully consider more contextual information when processing large-scale images, learn the positional relationship between objects, and thus more accurately identify the target area. Compared with the detection results of a simple variational autoencoder (VAE) proposal network, the recommended area of ​​the target distribution is significantly reduced due to the addition of image features.

[0028] (2) A multi-step detection architecture based on the Transformer region proposal network was designed. This step-by-step detection strategy can predict the area where the target may exist based on the rough detection results, and then perform detailed detection on the area, effectively improving the detection speed of large-scale remote sensing images and reducing the consumption of computing resources.

[0029] (3) A probability model-based evaluation index MG is proposed to accurately quantify the performance of the region proposal network. At the same time, through comparative experiments, the Transformer region proposal detection network proposed in the present invention is superior to the VAE-based proposal network in this evaluation index, indicating that the present invention can effectively remove the noise interference of the VAE model and estimate the position of small targets better than the VAE-type proposal network. In addition, compared with the conventional block detection method, the method of the present invention has achieved significant performance improvements in detection time, computing resource consumption, and the number of small targets detected. This fully demonstrates that the present invention is superior to other methods in terms of real-time detection and accuracy, and has considerable practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0031] Figure 1 is a flowchart of a remote sensing image step-by-step detection network based on Transformer region proposal according to an embodiment of the present invention;

[0032] Figure 2 2. It is a schematic diagram showing the difference between a variational autoencoder according to an embodiment of the present invention and a common autodecoder;

[0033] Figure 3 is a schematic diagram of a region proposal result according to an embodiment of the present invention;

[0034] Figure 4 is a schematic diagram of results based on dense detection according to an embodiment of the present invention;

[0035] Figure 5 2 is a schematic diagram of distribution detection results according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0037] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or inherent to these processes, methods, products or devices.

[0038] Example 1

[0039] According to an embodiment of the present invention, a remote sensing image step-by-step detection network based on Transformer region proposal is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system comprising at least one set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0040] Figure 1 is a flowchart of a remote sensing image step-by-step detection network based on Transformer region proposal according to an embodiment of the present invention. Figure 1 As shown, the network may include a first input layer, a label embedding layer, a feature fusion layer, a Transformer encoder, a Transformer decoder, and a fully connected layer; a second input layer, a variational autoencoder, an embedding layer, and a single attention layer. The network includes the following steps:

[0041] In step S101, the first input layer is connected to the label embedding layer, the label embedding layer is connected to the feature fusion layer, and the feature fusion layer is connected to the Transformer encoder; the second input layer is connected to the variational autoencoder, the variational autoencoder is connected to the embedding layer, the embedding layer is connected to the feature fusion layer, and the feature fusion layer is connected to the single attention layer; the single attention layer and the Transformer encoder are connected to the Transformer decoder, and the Transformer decoder is connected to the fully connected layer.

[0042] In this embodiment, if Figure 1 As shown, the single attention layer is a multi-head attention layer, and the fully connected layer is an MLP adapter. Figure 2 Schematic diagram of the difference between a variational autoencoder according to an embodiment of the present invention and a common autodecoder. Figure 2 Above the dotted line in is the encoding and decoding process of the ordinary autoencoder. Figure 2 Below the dotted line is the encoding and decoding process of the variational autoencoder. The first input layer is connected to the label embedding layer, the label embedding layer is connected to the feature fusion layer, the feature fusion layer is connected to the Transformer encoder as the first step detection network, the second input layer is connected to the variational autoencoder, the variational autoencoder is connected to the embedding layer, the embedding layer is connected to the feature fusion layer, and the feature fusion layer is connected to the single attention layer as the second step detection network.

[0043] In step S102, the first input layer obtains the original remote sensing image and inputs the original remote sensing image into the label embedding layer to generate a memory latent variable; the feature fusion layer adds the memory latent variable and the position code and inputs it into the Transformer encoder to generate a target latent variable.

[0044] In this embodiment, if Figure 1 As shown in the figure, the first input layer obtains the original remote sensing image, the label embedding layer processes the original remote sensing image to obtain the memory latent variable, and the memory latent variable is added to the position code and input into the Transformer encoder to obtain the target latent variable.

[0045] In step S103, the second input layer obtains the feature map of the large-scale target distribution corresponding to the original remote sensing image and inputs the feature map of the large-scale target distribution into the variational autoencoder to obtain a feature map of the Gaussian distribution; the feature map of the Gaussian distribution is input into the embedding layer to obtain a high-dimensional feature map, and the feature fusion layer adds the high-dimensional feature map to the position code to obtain a priori detection feature; after inputting the priori detection feature into the single attention layer, a target vector is generated, wherein the priori detection feature includes: an initial query vector, an initial key vector, and an initial value vector.

[0046] In this embodiment, if Figure 1As shown, the second input layer obtains the feature map of the large-scale target distribution corresponding to the original remote sensing image; the feature map of the large-scale target distribution is processed by a variational autoencoder to obtain a feature map of the Gaussian distribution; the feature map of the Gaussian distribution is input into the embedding layer to obtain a high-dimensional feature map, and the high-dimensional feature map is added to the position code to obtain the prior detection feature; after the prior detection feature is input into the single attention layer, a target vector is generated, where the prior detection feature includes: an initial query vector, an initial key vector, and an initial value vector.

[0047] In step S104, the target vector generated by the single attention layer and the target latent variable generated by the Transformer encoder are input into the Transformer decoder to obtain the shallow features of the original remote sensing image, and the shallow features of the original remote sensing image are input into the fully connected layer to obtain a small-size target two-dimensional distribution probability map of the original remote sensing image.

[0048] In this embodiment, if Figure 1 As shown in Figure 1, the target vector and target latent variable are input into the Transformer decoder to obtain the shallow features of the original remote sensing image, and the shallow features of the original remote sensing image are input into the fully connected layer to obtain the small-size target two-dimensional distribution probability map of the original remote sensing image.

[0049] The above method of this embodiment is further introduced below.

[0050] As an optional embodiment, in step S103, the expression of the prior detection feature is:

[0051]

[0052] in, is the initial query vector, is the initial key vector, is the initial value vector, is the feature map of the large-scale target distribution corresponding to the original remote sensing image, is a Gaussian distribution, is the position code, and its specific expression is , i is its sequence length, n is the feature dimension, k is a positive integer, is the convolution operation, It is the feature map of Gaussian distribution corresponding to the feature map of large-scale target distribution.

[0053] As an optional embodiment, in step S103, after inputting the prior detection features into the single attention layer, the expression for generating the target vector is:

[0054]

[0055] in, is the target query vector, is the query vector parameter, is the target key vector, key vector parameters, is the target value vector, is the value vector parameter;

[0056]

[0057] in, is the transpose, is the dimension of the K vector, is the target vector.

[0058] As an optional embodiment, the loss function of a remote sensing image step-by-step detection network based on Transformer region proposal is expressed as:

[0059]

[0060] in, is the loss function value, for and The cross entropy loss function value is, It is the small-size target two-dimensional distribution feature map of the original remote sensing image output by the step-by-step detection network. is the real two-dimensional distribution feature map of small-size targets in the original remote sensing image, KL is the regularization function, is random noise, The two-dimensional distribution feature map of small-sized targets in the original remote sensing image output by the step-by-step detection network corresponds to a normal distribution with a mean of , the variance is , is a standard normal distribution with a mean of 0 and a variance of 1.

[0061] In this embodiment, , is the adjustment factor, the value is [0,5], n is the number of detection categories, is the weight of each category, Predict the true target category label for the network probability.

[0062] As an optional embodiment, an evaluation indicator of a remote sensing image step-by-step detection network based on Transformer region proposal is the average slice sequence overlap. The process of determining the average slice sequence overlap is: obtaining the number of targets contained in each slice of the original remote sensing image, the probability value of each slice of the small-size target two-dimensional distribution feature map of the original remote sensing image, the initial number of slices and the total number of slices; sorting the number of targets contained in each slice from large to small to obtain a first sequence; sorting the probability value of each slice from large to small to obtain a second sequence; based on the initial number of slices, the first sequence corresponding to the initial number of slices and the second sequence corresponding to the initial number of slices, obtaining the overlap between the first sequence corresponding to the initial number of slices and the second sequence; based on the overlap between the first sequence corresponding to the initial number of slices and the second sequence and the total number of slices, obtaining the evaluation indicator of the second sequence.

[0063] In this embodiment, the original remote sensing image is divided into regions of the same size, each slice is a region, for example, the first slice includes 3 targets, the second slice includes 6 targets, and the probability value of each slice of the two-dimensional distribution feature map of small-size targets in the original remote sensing image is , And so on, sort the number of targets contained in each slice from large to small to obtain a first sequence sorted by the number of targets in each slice, sort the probability value of each slice from large to small to obtain a second sequence sorted by the probability value of each slice, calculate the initial number of slices, the first sequence corresponding to the initial number of slices, and the second sequence corresponding to the initial number of slices to obtain the overlap between the first sequence and the second sequence corresponding to the initial number of slices, calculate the overlap between the first sequence and the second sequence corresponding to the initial number of slices and the total number of slices to obtain the evaluation index of the second sequence.

[0064] As an optional embodiment, based on the initial number of slices, the first sequence corresponding to the initial number of slices, and the second sequence corresponding to the initial number of slices, an expression for the degree of overlap between the first sequence corresponding to the initial number of slices and the second sequence corresponding to the initial number of slices is obtained as follows:

[0065]

[0066] in, is the overlap between the first and second sequences corresponding to the initial number of slices, is the initial number of slices, is the first sequence corresponding to the initial number of slices, is the second sequence corresponding to the initial number of slices.

[0067] In this embodiment, for example, , that is, take the top 5 slices of the first sequence and the second sequence. When the number of the first slice target of the first sequence is the same as the number of the first slice target of the second sequence, is 1, when the first slice target number of the first sequence is different from the first slice target number of the first sequence, is 0, and so on, to get the total of the top 5 slices of the first and second sequences , Each slice Add and divide by the number of slices, that is .

[0068] As an optional embodiment, the expression for obtaining the evaluation index of the second sequence based on the overlap between the first sequence and the second sequence corresponding to the initial number of slices and the total number of slices is:

[0069]

[0070] in, is the evaluation index of the second sequence, is the total number of slices.

[0071] In this embodiment, for example, is 10, when Take different values ​​from 1 to 10 and calculate , accumulate and sum to obtain the evaluation index of the second sequence.

[0072] Experimental part:

[0073] The average slice sequence overlap MG is used as the evaluation index, and the sequence ratio is selected as 0.25, 0.5, and 1 respectively, and compared with the ordinary detection sequence without using the proposed network. The results are shown in Table 1:

[0074] Table 1 Evaluation indicators of different recommendation models (MG)

[0075]

[0076] As shown in Table 1, the evaluation metrics of our proposed method are all higher than those of both the non-region proposal network and the dense detection network. This demonstrates that our proposed network can learn the geographic location relationships of objects in the image and identify areas where small objects are more likely to be present based on the distribution of large objects. Our proposed network outperforms the proposed network using only VAE by 3.2%, 9.1%, and 4.8% when the evaluation metrics are 0.25, 0.5, and 1, respectively. This demonstrates that our model can effectively remove the noise interference of the VAE model, enabling more accurate estimation of small object locations.

[0077] On the other hand, the resource consumption comparison for large-size image detection is shown in Table 2:

[0078] Table 2 Comparison of resource consumption of different detection strategies

[0079]

[0080] As shown in Table 2, the method of the present invention is optimal in both detection time and FLOPS, and Params is suboptimal. Compared with conventional block detection, the present invention reduces computing resource consumption by 71.8%, and the detection time is shorter. At the same time, the number of small targets detected is close to 94.6%, which is 11.5 percentage points higher than the 83.1% of the dense detection method. The experimental results show that the present invention can effectively learn the spatial position relationship between targets and incorporate image features when inferring the proposed small target area, which makes the method of the present invention more suitable as a region proposal network to improve detection performance. In addition, the staged detection strategy adopted by the present invention can greatly reduce the detection time and reduce the consumption of computing resources when processing large-scale image detection tasks, thereby significantly improving the real-time performance of the detection task.

[0081] To further verify the effectiveness of the strategy of the present invention, an in-depth analysis is conducted through subjective comparison of remote sensing image detection results. Figure 3 is a schematic diagram of a region proposal result according to an embodiment of the present invention. Figure 3 (a) is the region recommendation result using VAE alone, while Figure 3 (b) is the region recommendation result of the present invention. As can be seen from the figure, the recommendation result is more accurate, and due to the addition of image features, the recommended area of ​​the target distribution is greatly reduced, which can effectively reduce computing resources.

[0082] Randomly select a large-scale remote sensing image and use dense detection results such as Figure 4 As shown, Figure 4 is a schematic diagram of the results based on dense detection according to an embodiment of the present invention, Figure 4 (a) is a large-scale remote sensing image, that is, the original image. Figure 4 (b) is the dense features and recommended areas corresponding to large-scale remote sensing images. Figure 4 (c) is the image to be detected in the second stage, Figure 4 (d) is the test result; Figure 5 2 is a schematic diagram of distribution detection results according to an embodiment of the present invention. Figure 5 (a) is a large-scale remote sensing image, that is, the original image. Figure 5 (b) is the one-stage detection result corresponding to the large-scale remote sensing image. Figure 5 (c) is the feature learning region proposal result corresponding to the large-scale remote sensing image. Figure 5(d) is the detection result corresponding to the large-scale remote sensing image. First, the source image is downsampled to 1024×1024 for a first-stage detection. The detector roughly detects the location of the large-scale targets in the image. The large-scale targets such as the port and the playground in the figure are accurately detected. Subsequently, the Transformer network is used to recommend areas where small targets may exist based on the location distribution information of the large targets, and a second-stage fine detection is performed. The detection results show that the detection frame has good wrapping properties for the target, and the boundary of the detection frame closely coincides with the target boundary box, verifying the effectiveness of the step-by-step detection based on feature learning. The two-stage detector detects the weak small targets that were not detected in the large-scale targets detected in the first stage, effectively reducing the waste of computing resources. Compared with the dense detection strategy, the Transformer network can provide recommended areas that are not within the range of large targets, such as Figure 5 As shown in (c), this recommendation is based on the potential spatial position relationship mapped out by high-dimensional abstract information, which can further improve the accuracy and efficiency of detection.

[0083] In summary, subjective and objective comparison experiments demonstrate that, compared to other methods, this method can more accurately detect faint targets through feature learning, significantly reduce detection time and computing resource consumption, and improve the real-time performance of detection tasks. These advantages give this invention significant application value and broad market prospects in the field of remote sensing image detection.

[0084] In the embodiment of the present invention, the first input layer is connected to the label embedding layer, the label embedding layer is connected to the feature fusion layer, and the feature fusion layer is connected to the Transformer encoder; the second input layer is connected to the variational autoencoder, the variational autoencoder is connected to the embedding layer, the embedding layer is connected to the feature fusion layer, and the feature fusion layer is connected to the single attention layer; the single attention layer and the Transformer encoder are connected to the Transformer decoder, and the Transformer decoder is connected to the fully connected layer; the first input layer obtains the original remote sensing image, inputs the original remote sensing image into the label embedding layer, and generates a memory latent variable; the feature fusion layer adds the memory latent variable and the position encoding and inputs it into the Transformer encoder to generate a target latent variable; the second input layer obtains the feature map of the large-scale target distribution corresponding to the original remote sensing image and inputs the feature map of the large-scale target distribution into the variational autoencoder to obtain a feature map of the Gaussian distribution; the feature map of the Gaussian distribution is input into the embedding layer to obtain a high-dimensional feature map, and the feature fusion layer adds the high-dimensional feature map to the position encoding. The codes are added to obtain the prior detection features; after the prior detection features are input into the single attention layer, a target vector is generated, wherein the prior detection features include: an initial query vector, an initial key vector and an initial value vector; the target vector generated by the single attention layer and the target latent variable generated by the Transformer encoder are input into the Transformer decoder to obtain the shallow features of the original remote sensing image, and the shallow features of the original remote sensing image are input into the fully connected layer to obtain a two-dimensional distribution probability map of small-size targets in the original remote sensing image. This solves the technical problems of existing remote sensing image target detection technology that is to process large-size images by reducing the input remote sensing image, or to crop the remote sensing image into smaller image slices and set the resampling rate to detect targets in the remote sensing image, resulting in long detection time, high resource utilization, loss of small target information and high missed detection rate. The technical effect of shortening the detection time of large-size remote sensing images as much as possible while ensuring detection accuracy is achieved by designing a Transformer region proposal step-by-step detection network based on feature learning.

[0085] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0086] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0087] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0088] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0089] In addition, the functional units in various embodiments of the present invention may be integrated into a first processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0090] The above are only preferred embodiments of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A remote sensing image step-by-step detection network based on Transformer region proposal, characterized by: include: First input layer, label embedding layer, feature fusion layer, Transformer encoder, Transformer decoder, fully connected layer; second input layer, variational autoencoder, embedding layer, single attention layer; The first input layer is connected to the label embedding layer, which is connected to the feature fusion layer, which is connected to the Transformer encoder. The second input layer is connected to the variational autoencoder, which is connected to the embedding layer, which is connected to the feature fusion layer, which is connected to the single attention layer. The single attention layer and the Transformer encoder are connected to the Transformer decoder, which is connected to the fully connected layer. The first input layer obtains the original remote sensing image and inputs the original remote sensing image into the label embedding layer to generate memory latent variables; The feature fusion layer adds the memory latent variable and the position code and inputs it into the Transformer encoder to generate the target latent variable; The second input layer obtains the feature map of the large-scale target distribution corresponding to the original remote sensing image and inputs the feature map of the large-scale target distribution into the variational autoencoder to obtain the feature map of the Gaussian distribution; the feature map of the Gaussian distribution is input into the embedding layer to obtain a high-dimensional feature map, and the feature fusion layer adds the high-dimensional feature map to the position code to obtain the prior detection feature; After inputting the prior detection features into the single attention layer, a target vector is generated, where the prior detection features include: initial query vector, initial key vector and initial value vector; The target vector generated by the single attention layer and the target latent variable generated by the Transformer encoder are input into the Transformer decoder to obtain the shallow features of the original remote sensing image. The shallow features of the original remote sensing image are input into the fully connected layer to obtain the small-size target two-dimensional distribution probability map of the original remote sensing image.

2. The network according to claim 1, wherein: The expression of the prior detection feature is: in, is the initial query vector, is the initial key vector, is the initial value vector, is the feature map of the large-scale target distribution corresponding to the original remote sensing image, is a Gaussian distribution, is the position code, and its specific expression is , i is the sequence length, n is the feature dimension, k is a positive integer, is the convolution operation, It is the feature map of Gaussian distribution corresponding to the feature map of large-scale target distribution.

3. The network according to claim 1, wherein: After inputting the prior detection features into the single attention layer, the expression for generating the target vector is: in, is the target query vector, is the query vector parameter, is the target key vector, key vector parameters, is the target value vector, is the value vector parameter; in, is the transpose, is the dimension of the K vector, is the target vector.

4. The network according to claim 1, wherein: The expression of the loss function is: in, is the loss function value, for and The cross entropy loss function value is, It is the small-size target two-dimensional distribution feature map of the original remote sensing image output by the step-by-step detection network. is the real two-dimensional distribution feature map of small-size targets in the original remote sensing image, KL is the regularization function, is random noise, The two-dimensional distribution feature map of small-sized targets in the original remote sensing image output by the step-by-step detection network corresponds to a normal distribution with a mean of , the variance is , is a standard normal distribution with a mean of 0 and a variance of 1.

5. The network according to claim 1, wherein: The evaluation index is the average slice sequence overlap. The process of determining the average slice sequence overlap is as follows: Obtain the number of targets contained in each slice of the original remote sensing image, the probability value of each slice of the small-size target two-dimensional distribution feature map of the original remote sensing image, the initial slice number and the total slice number; Sort the number of targets contained in each slice from large to small to obtain the first sequence; Sort the probability values ​​of each slice from large to small to obtain the second sequence; Obtaining a degree of overlap between the first sequence and the second sequence corresponding to the initial number of slices based on the initial number of slices, the first sequence corresponding to the initial number of slices, and the second sequence corresponding to the initial number of slices; Based on the overlap between the first sequence and the second sequence corresponding to the initial number of slices and the total number of slices, an evaluation index of the second sequence is obtained.

6. The network according to claim 5, characterized in that The expression for the degree of overlap between the first sequence corresponding to the initial number of slices and the second sequence corresponding to the initial number of slices is obtained based on the initial number of slices, the first sequence corresponding to the initial number of slices, and the second sequence corresponding to the initial number of slices: in, is the overlap between the first and second sequences corresponding to the initial number of slices, is the initial number of slices, is the first sequence corresponding to the initial number of slices, is the second sequence corresponding to the initial number of slices.

7. The network according to claim 5, characterized in that The expression for the evaluation index of the second sequence is obtained based on the overlap between the first sequence and the second sequence corresponding to the initial number of slices and the total number of slices: in, is the evaluation index of the second sequence, is the total number of slices.

8. A computer system, characterized in that include: One or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the network of claim 1.

9. A computer-readable storage medium, characterized in that Computer-executable instructions are stored, and when the instructions are executed, they are used to implement the network according to claim 1.

10. A computer program product, characterized in that The invention comprises computer executable instructions, which are used to implement the network of claim 1 when being executed.

Images