Remote sensing image step-by-step detection network based on Transform region suggestion

Abstract

The invention discloses a remote sensing image step-by-step detection network based on Transform region suggestion. The network comprises a first input layer, a label embedding layer, a feature fusion layer, a Transform encoder, a Transform decoder and a full connection layer, a second input layer, a variational auto-encoder, an embedded layer and a single attention layer; the first input layer is connected with the label embedding layer, the label embedding layer is connected with the feature fusion layer, and the feature fusion layer is connected with the Transform encoder. According to an existing remote sensing image target detection technology, a large-size image is processed by reducing an input remote sensing image, or the remote sensing image is cut into small image slices, and target detection in the remote sensing image is carried out by setting a re-collection rate; therefore, the technical problems of long detection time, more resource utilization, small target information loss and high omission ratio are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a remote sensing image step-by-step detection network based on Transformer region proposal. Background Art

[0002] In recent years, advances in deep learning technology have promoted significant developments in target detection tasks. Single-stage detection networks represented by the YOLO series use convolutional neural networks (CNNs) to directly extract features from the entire image and set a distribution grid in the detection head to predict the target information within the grid. This method significantly improves the detection speed and achieves comparable accuracy to other detection models without relying on the candidate region proposal mechanism, which plays an important role in the development of target detection in industrial applications. However, with the increase of model parameters, the CNN-based method has limited improvement in learning ability. To this end, an attention mechanism can be introduced to associate global features so that the model can learn more feature subspace information and improve the model's representation ability and decision accuracy.

[0003] In recent years, Transformer has been successfully introduced into computer vision from the field of natural language processing (NLP) and has achieved remarkable results. Unlike CNN, Transformer associates the global information of the image through the self-attention mechanism. In object detection, models such as DETR map the feature maps extracted by CNN into an embedded sequence, which is fed into the Transformer encoding and decoding module, and object detection is regarded as a query process. After the query and image features are calculated through the decoder for mutual attention, the FPN of the detection head outputs the location and category information of the target. The improved Deformable-DETR adopts a deformable multi-head attention mechanism, which allows the model to focus on specific salient areas in the image rather than the global one, thereby improving the convergence speed and processing efficiency, and significantly improving the detection ability of small objects in high-resolution images.

[0004] The above models have achieved good results in the field of target detection. However, existing target detection algorithms are mainly aimed at common images, while there are few studies on remote sensing images with large pixel sizes and small target proportions. Most methods crop the original high-resolution image into smaller image slices and set the resampling rate to prevent the target from being cut by the cropping line, and then train and detect the model. However, this method will lead to a geometric increase in detection time and resources. Other methods process large-size images by reducing the input image, which will cause the loss of small target information and increase the missed detection rate. Although these strategies are simple and direct, they will lead to serious detection errors or increased computational costs, which is not conducive to meeting the high requirements for real-time and safety in fields such as aviation, aerospace, and autonomous driving.

[0005] To solve the above problems, the present invention designs a Transformer region proposal step-by-step detection network based on feature learning, which can shorten the detection time of large-size remote sensing images as much as possible while ensuring the detection accuracy. Summary of the invention

[0006] The embodiment of the present invention provides a remote sensing image step-by-step detection network based on Transformer region proposal, which at least solves the technical problems that the existing remote sensing image target detection technology processes large-size images by reducing the input remote sensing image, or crops the remote sensing image into smaller image slices and sets the re-sampling rate to perform target detection in the remote sensing image, resulting in long detection time, high resource utilization, loss of small target information and high missed detection rate.

[0007] According to one aspect of an embodiment of the present invention, a remote sensing image step-by-step detection network based on Transformer region proposal is provided. The network may include: a first input layer, a label embedding layer, a feature fusion layer, a Transformer encoder, a Transformer decoder, and a fully connected layer; a second input layer, a variational autoencoder, an embedding layer, and a single attention layer; the first input layer is connected to the label embedding layer, the label embedding layer is connected to the feature fusion layer, and the feature fusion layer is connected to the Transformer encoder; the second input layer is connected to the variational autoencoder, the variational autoencoder is connected to the embedding layer, the embedding layer is connected to the feature fusion layer, and the feature fusion layer is connected to the single attention layer; the single attention layer and the Transformer encoder are connected to the Transformer decoder, and the Transformer decoder is connected to the fully connected layer; the first input layer obtains the original remote sensing image, inputs the original remote sensing image into the label embedding layer, and generates a memory latent variable; the feature fusion layer combines the memory latent variable with the position encoding The input is added to the Transformer encoder to generate target hidden variables; the second input layer obtains the feature map of the large-size target distribution corresponding to the original remote sensing image and inputs the feature map of the large-size target distribution into the variational autoencoder to obtain the feature map of the Gaussian distribution; the feature map of the Gaussian distribution is input into the embedding layer to obtain a high-dimensional feature map, and the feature fusion layer adds the high-dimensional feature map to the position code to obtain the prior detection feature; after the prior detection feature is input into the single attention layer, a target vector is generated, wherein the prior detection feature includes: an initial query vector, an initial key vector, and an initial value vector; the target vector generated by the single attention layer and the target hidden variable generated by the Transformer encoder are input into the Transformer decoder to obtain the shallow features of the original remote sensing image, and the shallow features of the original remote sensing image are input into the fully connected layer to obtain a two-dimensional distribution probability map of small-size targets of the original remote sensing image.

[0008] Optionally, the expression of the prior detection feature is:

[0009] in, is the initial query vector, is the initial key vector, is the initial value vector, is the characteristic map of large-size target distribution corresponding to the original remote sensing image, is a Gaussian distribution, is the position code, and its specific expression is , i is the sequence length, n is the feature dimension, k is a positive integer, is the convolution operation, It is the feature map of Gaussian distribution corresponding to the feature map of large-size target distribution.

[0010] Optionally, after the prior detection features are input into the single attention layer, the expression for generating the target vector is:

[0011] in, is the target query vector, is the query vector parameter, is the target key vector, key vector parameters, is the target value vector, is the value vector parameter;

[0012] in, For transposition, is the dimension of the K vector, is the target vector.

[0013] Alternatively, the loss function of a remote sensing image step-by-step detection network based on Transformer region proposal is expressed as:

[0014] in, is the loss function value, for and The cross entropy loss function value is, It is the two-dimensional distribution feature map of small-size targets in the original remote sensing image output by the step-by-step detection network. is the real two-dimensional distribution feature map of small-size targets in the original remote sensing image, KL is the regularization function, is random noise, The two-dimensional distribution feature map of small-size targets in the original remote sensing image output by the step-by-step detection network corresponds to a normal distribution, and its mean is , the variance is , is a standard normal distribution with a mean of 0 and a variance of 1.

[0015] Optionally, an evaluation index of a remote sensing image step-by-step detection network based on Transformer region proposal is an average slice sequence overlap, and a process for determining the average slice sequence overlap is: obtaining the number of targets contained in each slice of the original remote sensing image, the probability value of each slice of the small-size target two-dimensional distribution feature map of the original remote sensing image, the initial number of slices and the total number of slices; sorting the number of targets contained in each slice from large to small to obtain a first sequence; sorting the probability value of each slice from large to small to obtain a second sequence; based on the initial number of slices, the first sequence corresponding to the initial number of slices and the second sequence corresponding to the initial number of slices, obtaining the overlap between the first sequence corresponding to the initial number of slices and the second sequence; based on the overlap between the first sequence corresponding to the initial number of slices and the second sequence and the total number of slices, obtaining an evaluation index for the second sequence.

[0016] Optionally, the expression for obtaining the overlap between the first sequence corresponding to the initial number of slices and the second sequence corresponding to the initial number of slices based on the initial number of slices, the first sequence corresponding to the initial number of slices, and the second sequence corresponding to the initial number of slices is:

[0017] in, is the overlap between the first sequence and the second sequence corresponding to the initial number of slices, is the initial number of slices, is the first sequence corresponding to the initial number of slices, is the second sequence corresponding to the initial number of slices.

[0018] Optionally, the expression for obtaining the evaluation index of the second sequence based on the overlap between the first sequence and the second sequence corresponding to the initial number of slices and the total number of slices is:

[0019] is the evaluation index of the second sequence, is the total number of slices.

[0020] Beneficial effects of the present invention: (1) The adopted Transformer model can capture the global information in the image through the self-attention mechanism. This feature enables the model to fully consider more contextual information when processing large-size images, learn the positional relationship between objects, and thus more accurately identify the target area. Compared with the detection results of the simple variational autoencoder (VAE) proposal network, the recommended area of ​​the target distribution is greatly reduced due to the addition of image features.

[0021] (2) A multi-step detection architecture based on the Transformer region proposal network was designed. This step-by-step detection strategy can predict the area where the target may exist based on the rough detection results, and then perform detailed detection on the area, effectively improving the detection speed of large-scale remote sensing images and reducing the consumption of computing resources.

[0022] (3) A probability model-based evaluation index MG is proposed to accurately quantify the performance of the region proposal network. At the same time, through comparative experiments, the Transformer region proposal detection network proposed in the present invention is higher than the VAE-based proposal network in this evaluation index, indicating that the present invention can effectively remove the noise interference of the VAE model and estimate the position of small targets better than the VAE-type proposal network. In addition, compared with the conventional block detection method, the method of the present invention has achieved significant performance improvements in detection time, computing resource consumption, and the number of small targets detected. This fully demonstrates that the present invention is superior to other methods in terms of real-time and accuracy of detection, and has considerable practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 is a flowchart of a remote sensing image step-by-step detection network based on Transformer region proposal according to an embodiment of the present invention; Figure 2 is a schematic diagram of the difference between a variational autoencoder and a common autodecoder according to an embodiment of the present invention; Figure 3 is a schematic diagram of a region proposal result according to an embodiment of the present invention; Figure 4 is a schematic diagram of a result based on dense detection according to an embodiment of the present invention; Figure 5 4 is a schematic diagram of distribution detection results according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] Example 1 According to an embodiment of the present invention, a remote sensing image step-by-step detection network based on Transformer region proposal is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system comprising at least one set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0027] Figure 1 is a flowchart of a remote sensing image step-by-step detection network based on Transformer region proposal according to an embodiment of the present invention. Figure 1 As shown, the network may include a first input layer, a label embedding layer, a feature fusion layer, a Transformer encoder, a Transformer decoder, and a fully connected layer; a second input layer, a variational autoencoder, an embedding layer, and a single attention layer. The network includes the following steps: Step S101, the first input layer is connected to the label embedding layer, the label embedding layer is connected to the feature fusion layer, the feature fusion layer is connected to the Transformer encoder; the second input layer is connected to the variational autoencoder, the variational autoencoder is connected to the embedding layer, the embedding layer is connected to the feature fusion layer, the feature fusion layer is connected to the single attention layer; the single attention layer and the Transformer encoder are connected to the Transformer decoder, and the Transformer decoder is connected to the fully connected layer.

[0028] In this embodiment, if Figure 1 As shown, the single attention layer is a multi-head attention layer, and the fully connected layer is an MLP adapter. Figure 2 is a schematic diagram showing the difference between a variational autoencoder according to an embodiment of the present invention and a common autodecoder, Figure 2 Above the dotted line in is the encoding and decoding process of the ordinary autoencoder. Figure 2 Below the dotted line is the encoding and decoding process of the variational autoencoder. The first input layer is connected to the label embedding layer, the label embedding layer is connected to the feature fusion layer, the feature fusion layer is connected to the Transformer encoder as the first step detection network, the second input layer is connected to the variational autoencoder, the variational autoencoder is connected to the embedding layer, the embedding layer is connected to the feature fusion layer, and the feature fusion layer is connected to the single attention layer as the second step detection network.

[0029] In step S102, the first input layer obtains the original remote sensing image, inputs the original remote sensing image into the label embedding layer, and generates a memory latent variable; the feature fusion layer adds the memory latent variable and the position code and inputs it into the Transformer encoder to generate a target latent variable.

[0030] In this embodiment, if Figure 1 As shown in the figure, the first input layer obtains the original remote sensing image, the label embedding layer processes the original remote sensing image to obtain the memory latent variable, and the memory latent variable is added to the position code and input into the Transformer encoder to obtain the target latent variable.

[0031] In step S103, the second input layer obtains the feature map of the large-size target distribution corresponding to the original remote sensing image and inputs the feature map of the large-size target distribution into the variational autoencoder to obtain the feature map of the Gaussian distribution; the feature map of the Gaussian distribution is input into the embedding layer to obtain a high-dimensional feature map, and the feature fusion layer adds the high-dimensional feature map to the position code to obtain a priori detection feature; after the priori detection feature is input into the single attention layer, a target vector is generated, wherein the priori detection feature includes: an initial query vector, an initial key vector and an initial value vector.

[0032] In this embodiment, if Figure 1 As shown, the second input layer obtains the feature map of the large-size target distribution corresponding to the original remote sensing image; the feature map of the large-size target distribution is processed by a variational autoencoder to obtain a feature map of a Gaussian distribution; the feature map of the Gaussian distribution is input into the embedding layer to obtain a high-dimensional feature map, and the high-dimensional feature map is added to the position code to obtain a priori detection feature; after the prior detection feature is input into the single attention layer, a target vector is generated, wherein the prior detection feature includes: an initial query vector, an initial key vector, and an initial value vector.

[0033] In step S104, the target vector generated by the single attention layer and the target latent variable generated by the Transformer encoder are input into the Transformer decoder to obtain the shallow features of the original remote sensing image, and the shallow features of the original remote sensing image are input into the fully connected layer to obtain a two-dimensional distribution probability map of small-size targets of the original remote sensing image.

[0034] In this embodiment, if Figure 1 As shown in the figure, the target vector and the target latent variable are input into the Transformer decoder to obtain the shallow features of the original remote sensing image, and the shallow features of the original remote sensing image are input into the fully connected layer to obtain the two-dimensional distribution probability map of the small-size target of the original remote sensing image.

[0035] The above method of this embodiment is further introduced below.

[0036] As an optional implementation manner, in step S103, the expression of the prior detection feature is:

[0037] in, is the initial query vector, is the initial key vector, is the initial value vector, is the characteristic map of large-size target distribution corresponding to the original remote sensing image, is a Gaussian distribution, is the position code, and its specific expression is , i is the sequence length, n is the feature dimension, k is a positive integer, is the convolution operation, It is the feature map of Gaussian distribution corresponding to the feature map of large-size target distribution.

[0038] As an optional implementation method, in step S103, after the prior detection feature is input into the single attention layer, the expression for generating the target vector is:

[0039] in, is the target query vector, is the query vector parameter, is the target key vector, key vector parameters, is the target value vector, is the value vector parameter;

[0040] in, For transposition, is the dimension of the K vector, is the target vector.

[0041] As an optional implementation method, the loss function of a remote sensing image step-by-step detection network based on Transformer region proposal is expressed as:

[0042] in, is the loss function value, for and The cross entropy loss function value is, It is the two-dimensional distribution feature map of small-size targets in the original remote sensing image output by the step-by-step detection network. is the real two-dimensional distribution feature map of small-size targets in the original remote sensing image, KL is the regularization function, is random noise, The two-dimensional distribution feature map of small-size targets in the original remote sensing image output by the step-by-step detection network corresponds to a normal distribution, and its mean is , the variance is , is a standard normal distribution with a mean of 0 and a variance of 1.

[0043] In this embodiment, , is the adjustment factor, the value is [0,5], n is the number of detection categories, is the weight of each category, Predict the true target category label for the network probability.

[0044] As an optional implementation method, an evaluation index of a remote sensing image step-by-step detection network based on Transformer region proposal is the average slice sequence overlap, and the process of determining the average slice sequence overlap is: obtaining the number of targets contained in each slice of the original remote sensing image, the probability value of each slice of the small-size target two-dimensional distribution feature map of the original remote sensing image, the initial number of slices and the total number of slices; sorting the number of targets contained in each slice from large to small to obtain a first sequence; sorting each slice probability value from large to small to obtain a second sequence; based on the initial number of slices, the first sequence corresponding to the initial number of slices and the second sequence corresponding to the initial number of slices, obtaining the overlap of the first sequence corresponding to the initial number of slices and the second sequence; based on the overlap of the first sequence corresponding to the initial number of slices and the second sequence and the total number of slices, obtaining the evaluation index of the second sequence.

[0045] In this embodiment, the original remote sensing image is divided into regions of the same size, each slice is a region, for example, the first slice includes 3 targets, the second slice includes 6 targets, and the probability value of each slice of the two-dimensional distribution feature map of small-size targets in the original remote sensing image is , And so on, sort the number of targets contained in each slice from large to small, and obtain the first sequence sorted by the number of targets in each slice, sort the probability value of each slice from large to small, and obtain the second sequence sorted by the probability value of each slice, calculate the initial number of slices, the first sequence corresponding to the initial number of slices, and the second sequence corresponding to the initial number of slices to obtain the overlap between the first sequence and the second sequence corresponding to the initial number of slices, calculate the overlap between the first sequence and the second sequence corresponding to the initial number of slices and the total number of slices, and obtain the evaluation index of the second sequence.

[0046] As an optional implementation manner, the expression for obtaining the overlap between the first sequence corresponding to the initial number of slices and the second sequence corresponding to the initial number of slices based on the initial number of slices, the first sequence corresponding to the initial number of slices, and the second sequence corresponding to the initial number of slices is:

[0047] in, is the overlap between the first sequence and the second sequence corresponding to the initial number of slices, is the initial number of slices, is the first sequence corresponding to the initial number of slices, is the second sequence corresponding to the initial number of slices.

[0048] In this embodiment, for example, , that is, take the top 5 slices of the first sequence and the second sequence. When the number of first slice targets of the first sequence is the same as the number of first slice targets of the second sequence, is 1, when the first slice target number of the first sequence is different from the first slice target number of the first sequence, is 0, and so on, to get the total of the top 5 slices in the first and second sequences , Each slice Add and divide by the number of slices, that is .

[0049] As an optional implementation manner, the expression for obtaining the evaluation index of the second sequence based on the overlap degree of the first sequence and the second sequence corresponding to the initial number of slices and the total number of slices is:

[0050] in, is the evaluation index of the second sequence, is the total number of slices.

[0051] In this embodiment, for example, is 10, when Take different values ​​from 1 to 10 and calculate , accumulate and sum to obtain the evaluation index of the second sequence.

[0052] Experimental part: The average slice sequence overlap MG is used as the evaluation index, and the sequence ratio is selected as 0.25, 0.5, and 1 respectively, and compared with the common detection sequence without using the proposed network. The results are shown in Table 1: Table 1 Evaluation indicators of different recommendation models (MG)

[0053] As shown in Table 1, the evaluation indexes of the present invention are higher than those of the non-region proposal network and the dense detection network, which shows that the proposal network of the present invention can learn the geographical location relationship of the target in the image and find the area where small targets are more likely to exist based on the distribution of large targets. The present invention is 3.2%, 9.1%, and 4.8% higher than the proposal network using only VAE when the evaluation index is 0.25, 0.5, and 1, respectively, proving that the model can effectively remove the noise interference of the VAE model, making it more accurate in estimating the position of small targets.

[0054] On the other hand, the resource consumption comparison of large-size image detection is shown in Table 2: Table 2 Comparison of resource consumption of different detection strategies

[0055] As shown in Table 2, the method of the present invention is optimal in both detection time and FLOPS, and Params is suboptimal. Compared with conventional block detection, the present invention reduces the computing resource consumption by 71.8%, and the detection time is shorter. At the same time, the number of small targets detected is close to 94.6%, which is 11.5 percentage points higher than 83.1% of the dense detection method. The experimental results show that the present invention can effectively learn the spatial position relationship between targets and incorporate image features when inferring the proposed small target area, which makes the method of the present invention more suitable as a region proposal network to improve detection performance. In addition, the staged detection strategy adopted by the present invention can significantly reduce the detection time and reduce the consumption of computing resources when processing large-scale image detection tasks, thereby significantly improving the real-time performance of the detection task.

[0056] To further verify the effectiveness of the strategy of the present invention, an in-depth analysis is conducted through subjective comparison of remote sensing image detection results. Figure 3is a schematic diagram of a region proposal result according to an embodiment of the present invention, Figure 3 (a) is the region recommendation result using VAE alone, while Figure 3 (b) is the region recommendation result of the present invention. As can be seen from the figure, the recommendation result is more accurate, and due to the addition of image features, the recommended region of the target distribution is greatly reduced, which can effectively reduce computing resources.

[0057] Randomly select a large-size remote sensing image and use dense detection results such as Figure 4 As shown, Figure 4 is a schematic diagram of a result based on dense detection according to an embodiment of the present invention, Figure 4 (a) is a large-size remote sensing image, that is, the original image. Figure 4 (b) is the dense features and recommended regions corresponding to large-scale remote sensing images. Figure 4 (c) is the image to be detected in the second stage. Figure 4 (d) is the test result; Figure 5 4 is a schematic diagram of distribution detection results according to an embodiment of the present invention. Figure 5 (a) is a large-size remote sensing image, that is, the original image. Figure 5 (b) is the one-stage detection result corresponding to the large-size remote sensing image. Figure 5 (c) is the feature learning region proposal result corresponding to the large-scale remote sensing image. Figure 5 (d) is the detection result corresponding to the large-scale remote sensing image. First, the source image is downsampled to 1024×1024 for a first-stage detection. The detector roughly detects the location of the large-scale targets in the image. The large-scale targets such as the port and the playground in the figure are accurately detected. Subsequently, the Transformer network is used to recommend areas where small targets may exist based on the location distribution information of the large targets, and a second-stage fine detection is performed. The detection results show that the detection box has good wrapping properties for the target, and the boundary of the detection box closely overlaps with the target boundary box, which verifies the effectiveness of step-by-step detection based on feature learning. The two-stage detector detects small and weak targets that were not detected in the large-scale targets detected in the first stage, effectively reducing the waste of computing resources. Compared with dense detection strategies, the Transformer network can provide recommended areas that are not within the range of large targets, such as Figure 5 As shown in (c), this recommendation is based on the potential spatial position relationship mapped out by high-dimensional abstract information, which can further improve the accuracy and efficiency of detection.

[0058] In summary, the subjective and objective comparison experiments show that compared with other methods, it can rely on feature learning to detect faint targets more accurately, greatly reduce detection time and computing resource consumption, and improve the real-time performance of detection tasks. These advantages make the present invention have important application value and broad market prospects in the field of remote sensing image detection.

[0059] In the embodiment of the present invention, the first input layer is connected to the label embedding layer, the label embedding layer is connected to the feature fusion layer, and the feature fusion layer is connected to the Transformer encoder; the second input layer is connected to the variational autoencoder, the variational autoencoder is connected to the embedding layer, the embedding layer is connected to the feature fusion layer, and the feature fusion layer is connected to the single attention layer; the single attention layer and the Transformer encoder are connected to the Transformer decoder, and the Transformer decoder is connected to the fully connected layer; the first input layer obtains the original remote sensing image, inputs the original remote sensing image into the label embedding layer, and generates a memory latent variable; the feature fusion layer adds the memory latent variable and the position encoding and inputs it into the Transformer encoder to generate a target latent variable; the second input layer obtains the feature map of the large-size target distribution corresponding to the original remote sensing image and inputs the feature map of the large-size target distribution into the variational autoencoder to obtain a feature map of the Gaussian distribution; the feature map of the Gaussian distribution is input into the embedding layer to obtain a high-dimensional feature map, and the feature fusion layer adds the high-dimensional feature map to the position encoding. The codes are added to obtain the prior detection features; after the prior detection features are input into the single attention layer, a target vector is generated, wherein the prior detection features include: an initial query vector, an initial key vector and an initial value vector; the target vector generated by the single attention layer and the target latent variable generated by the Transformer encoder are input into the Transformer decoder to obtain the shallow features of the original remote sensing image, and the shallow features of the original remote sensing image are input into the fully connected layer to obtain a two-dimensional distribution probability map of small-size targets in the original remote sensing image, which solves the problem that the existing remote sensing image target detection technology processes large-size images by reducing the input remote sensing image, or crops the remote sensing image into smaller image slices, sets the resampling rate to detect targets in the remote sensing image, resulting in long detection time, high resource utilization, small target information loss and high missed detection rate. The technical effect of shortening the detection time of large-size remote sensing images as much as possible while ensuring the detection accuracy is achieved by designing a Transformer region proposal step-by-step detection network based on feature learning.

[0060] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0061] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0062] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0063] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0064] In addition, each functional unit in each embodiment of the present invention may be integrated into a first processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0065] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A remote sensing image step-by-step detection network based on Transformer region proposal, characterized in that: include: First input layer, label embedding layer, feature fusion layer, Transformer encoder, Transformer decoder, fully connected layer; second input layer, variational autoencoder, embedding layer, single attention layer; The first input layer is connected to the label embedding layer, which is connected to the feature fusion layer, which is connected to the Transformer encoder; the second input layer is connected to the variational autoencoder, which is connected to the embedding layer, which is connected to the feature fusion layer, which is connected to the single attention layer; the single attention layer and the Transformer encoder are connected to the Transformer decoder, which is connected to the fully connected layer; The first input layer obtains the original remote sensing image, inputs the original remote sensing image into the label embedding layer, and generates memory latent variables; The feature fusion layer adds the memory latent variable and the position code and inputs it into the Transformer encoder to generate the target latent variable. The second input layer obtains the feature map of the large-size target distribution corresponding to the original remote sensing image and inputs the feature map of the large-size target distribution into the variational autoencoder to obtain the feature map of the Gaussian distribution; the feature map of the Gaussian distribution is input into the embedding layer to obtain a high-dimensional feature map, and the feature fusion layer adds the high-dimensional feature map to the position code to obtain the prior detection feature; After the prior detection features are input into the single attention layer, a target vector is generated, wherein the prior detection features include: an initial query vector, an initial key vector, and an initial value vector; The target vector generated by the single attention layer and the target latent variable generated by the Transformer encoder are input into the Transformer decoder to obtain the shallow features of the original remote sensing image. The shallow features of the original remote sensing image are input into the fully connected layer to obtain the two-dimensional distribution probability map of small-size targets in the original remote sensing image.

2. The network according to claim 1, characterized in that The expression of the prior detection feature is: in, is the initial query vector, is the initial key vector, is the initial value vector, is the characteristic map of large-size target distribution corresponding to the original remote sensing image, is a Gaussian distribution, is the position code, and its specific expression is , i is the sequence length, n is the feature dimension, k is a positive integer, is the convolution operation, It is the feature map of Gaussian distribution corresponding to the feature map of large-size target distribution.

3. The network according to claim 1, characterized in that After the prior detection features are input into the single attention layer, the expression for generating the target vector is: in, is the target query vector, is the query vector parameter, is the target key vector, key vector parameters, is the target value vector, is the value vector parameter; in, For transposition, is the dimension of the K vector, is the target vector.

4. The network according to claim 1, characterized in that The expression of the loss function is: in, is the loss function value, for and The cross entropy loss function value is, It is the two-dimensional distribution feature map of small-size targets in the original remote sensing image output by the step-by-step detection network. is the real two-dimensional distribution feature map of small-size targets in the original remote sensing image, KL is the regularization function, is random noise, The two-dimensional distribution feature map of small-size targets in the original remote sensing image output by the step-by-step detection network corresponds to a normal distribution, and its mean is , the variance is , is a standard normal distribution with a mean of 0 and a variance of 1.

5. The network according to claim 1, characterized in that The evaluation index is the average slice sequence overlap, and the process of determining the average slice sequence overlap is: Obtain the number of targets contained in each slice of the original remote sensing image, the probability value of each slice of the two-dimensional distribution feature map of small-size targets in the original remote sensing image, the number of initial slices and the total number of slices; Sort the number of targets contained in each slice from large to small to obtain the first sequence; Sort the probability value of each slice from large to small to obtain a second sequence; Based on the initial number of slices, the first sequence corresponding to the initial number of slices, and the second sequence corresponding to the initial number of slices, obtaining a degree of overlap between the first sequence corresponding to the initial number of slices and the second sequence; Based on the overlap between the first sequence and the second sequence corresponding to the initial number of slices and the total number of slices, an evaluation index of the second sequence is obtained.

6. The network according to claim 5, characterized in that The expression for the overlap between the first sequence corresponding to the initial number of slices and the second sequence corresponding to the initial number of slices is obtained based on the initial number of slices, the first sequence corresponding to the initial number of slices, and the second sequence corresponding to the initial number of slices: in, is the overlap between the first sequence and the second sequence corresponding to the initial number of slices, is the initial number of slices, is the first sequence corresponding to the initial number of slices, is the second sequence corresponding to the initial number of slices.

7. The network according to claim 5, characterized in that The expression for obtaining the evaluation index of the second sequence based on the overlap between the first sequence and the second sequence corresponding to the initial number of slices and the total number of slices is: in, is the evaluation index of the second sequence, is the total number of slices.

8. A computer system, characterized in that include: One or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the network of claim 1.

9. A computer-readable storage medium, characterized in that Computer executable instructions are stored, and when the instructions are executed, they are used to implement the network of claim 1.

10. A computer program product, characterized in that The invention comprises computer executable instructions, which are used to implement the network of claim 1 when being executed.

Images