Remote sensing multi-scale target detection network method based on target scale explicit adjustment

Abstract

The invention discloses a remote sensing multi-scale target detection network method based on target scale explicit adjustment. The method comprises the following steps: calculating a feature map F1 of an input image I; generating an anchor frame A; distributing an explicit amplification anchor frame A * for the proposal generation network; the explicit amplification learning layer generates G2 *; generating a region proposal P; distributing an explicit amplification proposal P * for the detection network; sampling a region of interest (FP); the explicit recovery learning layer generates G4 *; and the detection network predicts a target detection result D. According to the remote sensing multi-scale target detection network method based on target scale explicit adjustment and the moving and driving method, the problem that an existing method is difficult to display and research a small target due to the fact that the target is more sensitive to bounding box offset and few target pixels cause less information amount and are not beneficial to judgment is solved.

Description

Technical Field

[0001] The present invention belongs to the field of information processing technology, and in particular relates to a remote sensing multi-scale target detection network method based on explicit adjustment of target scale. Background Art

[0002] High-resolution remote sensing target detection is one of the cutting-edge research directions in the current field of remote sensing technology. It uses high-resolution satellite remote sensing data and advanced image processing and computer vision technology to achieve fine identification of objects and target detection. High-resolution remote sensing target detection technology has broad application prospects in environmental protection, urban planning, resource investigation, military reconnaissance and other fields. With the continuous development of remote sensing technology, high-resolution remote sensing target detection is also constantly improving in accuracy and reliability, becoming one of the important development directions of remote sensing technology.

[0003] With the development of deep learning in recent years, researchers have proposed a large number of remote sensing image neural network algorithms, which have significantly improved the average accuracy of high-resolution remote sensing target detection. However, the target detection algorithm based on deep learning has encountered the problem of small target detection difficulties in remote sensing images. This problem arises from two points: the large-scale characteristics of remote sensing images require the neural network to perform preprocessing such as downsampling on the input image; during the downsampling process, the number of pixels of many targets is compressed to a very limited level, making many targets difficult to detect and identify. Therefore, the problem of small target detection difficulties in remote sensing images is essentially a contradiction between detection efficiency and detection accuracy. In response to the problem of small target detection difficulties, some representative works are as follows:

[0004] The advent of generative adversarial networks (GANs) has opened up a new possibility for data augmentation for training architectures without the expensive task of annotating huge datasets for small objects. In “A full data augmentation pipeline for small object detection based on generative adversarial networks”, Brais et al. proposed a complete pipeline for data augmentation for small object detection that combines a GAN-based object generator with object segmentation, image inpainting, and image blending techniques to achieve high-quality synthetic data. This approach can be widely used in various methods through data augmentation.

[0005] When the data set is already determined, we can also add a setting strategy for the anchors responsible for small targets to make the learning of small targets more comprehensive during training. For example, Zhang et al. proposed FaceBoxes in the paper "Faceboxes: A CPU real-time and accurate unconstrained face detector". The method in this paper sets different numbers of anchors for targets of different scales to ensure the same anchor density, thereby ensuring the recall rate of small targets. Obviously, this method needs to increase the amount of calculation to improve the detection effect of small targets.

[0006] Small objects, especially objects like faces, do not appear alone in images. Tang et al. proposed in "PyramidBox: A Context-assisted Single Shot Face Detector" that adding some contextual information such as head and shoulders will make the object larger, and detection will be easier with contextual information. Unfortunately, this paper did not conduct in-depth research on the explicit expression of this context.

[0007] In summary, none of the above methods explicitly studies the small scale of small objects. The difficulty in detecting small objects is greatly affected by the following two factors: 1) small objects are more sensitive to bounding box offsets; 2) small objects have fewer pixels, resulting in less information, which is not conducive to discrimination. Summary of the invention

[0008] The purpose of the present invention is to provide a remote sensing multi-scale target detection network method based on explicit adjustment of the target scale. The existing methods are difficult to display and study small targets because the target is more sensitive to bounding box offset and the target has fewer pixels, resulting in less information and unfavorable for discrimination.

[0009] In order to achieve the above object, the technical solution adopted by the present invention is: a remote sensing multi-scale target detection network method based on explicit adjustment of target scale, which is specifically implemented according to the following steps:

[0010] Step 1, calculate the feature map F1 of the input image I;

[0011] Step 2: Generate anchor box A;

[0012] Step 3: Assign explicit enlarged anchor boxes A to the proposal generation network * ;

[0013] Step 4: Explicitly enlarge the learning layer to generate G2 * ;

[0014] Step 5: Generate region proposal P;

[0015] Step 6: Assign explicit upscale proposals P to the detection network * ;

[0016] Step 7: Sampling the region of interest F P ;

[0017] Step 8: Explicitly restore the learning layer to generate G4 * ;

[0018] Step 9: The detection network predicts the target detection result D.

[0019] As a preferred technical solution of the present invention, in step 1, the input image I is processed by a network with several layers of structure to obtain a feature map F1.

[0020] As a preferred technical solution of the present invention, in step 2, generating the anchor frame A is specifically as follows:

[0021] Input the preset anchor box area S = {s1, a2, ..., s K}、Preset aspect ratio parameter R={r1,r2,…,r L}; Combine the two parameters and output a set of rectangular boxes with different areas and aspect ratios at intervals of several pixels in the x and y directions A={a1,a2,…,a N}, N = K*L; output multiple groups of anchor boxes A(i,j), where i,j represent pixel positions.

[0022] As a preferred technical solution of the present invention, in step 3, an explicit enlarged anchor box A is assigned to the proposal generation network. * Specifically:

[0023] Input a set of anchor boxes A = {a1, a2, ..., a n ,…,a N}, and the true target boxes G = {g1,g2,…,g m ,…,g M}; Output the final target box positive sample A * ={a 1* ,a 2* ,…,a n* ,…,a N1*}, the length is changed to N1;

[0024] Enlarge the smaller box labels in G by a specific ratio r1 to obtain G1 * = {g 1 1,g 1 2,…,g 1 m ,…,g 1 M};

[0025] The threshold s1 used for judging "small size" can be set as an absolute size threshold or a relative size threshold;

[0026] The "specific ratio r1" is set to 2;

[0027] Then A and G1 * Perform one-to-one matching and try to find the n Find matching g 1 m If no match is found for g 1 m Then delete a from A n ;

[0028] Here, “matching” means that the overlap evaluation index of the two frames is greater than the threshold.

[0029] As a preferred technical solution of the present invention, in step 4, generating the display magnification learning layer is specifically as follows:

[0030] Input the true target box G of all targets; enlarge the smaller box labels in G by a specific ratio r2 to obtain G2 * = {g 2 1,g 2 2,…,g 2 m ,…,g 2 M}; Output explicit enlarged learning target box G2 * ;

[0031] The threshold s2 used for judging "small size" can be set as an absolute size threshold or a relative size threshold;

[0032] The “specific ratio r2” is set to 2.

[0033] As a preferred technical solution of the present invention, in step 5, generating the region proposal P is specifically as follows:

[0034] Input feature map F1, target box positive sample pair A * , explicitly enlarge the learning target box G2 * ; Output region proposal P = {p1,p2,…,p o ,…,p O};

[0035] For each a n : Predict the target box a in the feature map F1 n The corresponding area is sampled and scaled to the set size to obtain R;

[0036] Perform multi-layer convolution or full connection on R, and output region proposal P = {p1, p2, ..., p o* …,p O};

[0037] In the training process of the model, the real target box used by the objective function of "multi-layer convolution or full connection" is G2 * .

[0038] As a preferred technical solution of the present invention, in step 6, an explicit amplification proposal P is allocated to the detection network. * , specifically:

[0039] Input region proposal P, true target box G of all targets; output explicit enlarged proposal positive sample P * ={p1*,p2*,…,p o *,…,p O *};

[0040] Enlarge the smaller box labels in G by a specific ratio r3 to obtain G3 * = {g 3 1,g 3 2,…,g 3 m ,…,g 3 M};

[0041] The threshold s3 used for judging "small size" can be set as an absolute size threshold or a relative size threshold;

[0042] The "specific ratio r3" is set to 2;

[0043] Then P and G3 * Perform one-to-one matching and try to find the n Find matching g 3 m If no match is found for g 3 m Then delete p from P n , get P * .

[0044] As a preferred technical solution of the present invention, in step 7, the region of interest sampling F P Specifically:

[0045] Input feature map F1, explicitly enlarge the proposal positive sample P * : Output feature map F of the region of interest P ={F p1* ,F p2* ,…,F po* ,…,F pO*}:

[0046] For each p o *: Predict the target box p in the feature map F1 o * The corresponding area is sampled and scaled to the set size to obtain F po* .

[0047] As a preferred technical solution of the present invention, in step 8, the explicit restoration learning layer generates G4 * Specifically:

[0048] Input the true target box G of all targets; enlarge the smaller box labels in G by a specific ratio r4 to obtain G4 * = {g 2 1,g 2 2,…,g 2 m ,…,g 2 M}; Output explicit restoration learning target frame G4 * ;

[0049] The threshold s4 used for judging "small size" can be set as an absolute size threshold or a relative size threshold;

[0050] The “specific ratio r4” is set to 1.

[0051] As a preferred technical solution of the present invention, in step 9, the detection network predicts the target detection result D as follows:

[0052] Input the feature map F of the region of interest P , explicitly enlarge the learning target box G4 * ; for F P Perform multi-layer convolution or full connection; output detection result D = {d1, d2, ..., d h* …,d H};

[0053] In the training process of the model, the real target box used by the objective function of "multi-layer convolution or full connection" is G4 * .

[0054] The beneficial effects of the present invention are:

[0055] (1) The remote sensing multi-scale target detection network method based on explicit adjustment of the target scale of the present invention can effectively improve the recall rate of small targets and the accuracy of multi-size target detection by improving the region proposal generation network, the input box allocation of the detector, and the output box learning target. In order to implement this model, a network for target detection is built. This model is applicable to all two-stage and multi-stage detection network methods, such as Faster R-CNN, Cascade R-CNN and other networks based on these networks. In addition, the proposed method achieves an improvement in detection effect without increasing the amount of calculation as much as possible;

[0056] (2) The remote sensing multi-scale target detection network method based on explicit adjustment of the target scale of the present invention can improve the recall rate of small targets through improvements in the region proposal generation network. First, when assigning the input anchor frame to the true target frame, if the true target frame size is smaller than the set size (for example, 32*32), the true target frame is enlarged to match the larger anchor frame; secondly, when training the region proposal generation network, if the true target frame size is smaller than the set size (for example, 32*32), the true target frame is also enlarged, and the enlarged target frame is used as the learning target. The proposed method can process small targets at a larger size, avoiding the difficulty brought to the detection task by the small size itself (that is, small targets are more sensitive to the offset of the bounding box);

[0057] (3) The remote sensing multi-scale target detection network method based on explicit adjustment of target scale of the present invention, first, when assigning the input region proposal to the real target frame, if the real target size is smaller than the set size (for example, 32*32), the real target frame is enlarged to match the larger anchor frame; in addition, when training the detector, the real target frame of the small target is directly used as the learning target. The proposed method makes the input proposal size slightly larger than the small target itself, providing a certain fault tolerance to ensure that the small target is within the frame, thereby better retaining the pixel information of the small target itself, and using more target information to restore the final position of the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0059] Figure 1 It is a schematic diagram of the remote sensing multi-scale target detection network method based on explicit adjustment of target scale of the present invention. DETAILED DESCRIPTION

[0060] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0061] Example 1

[0062] like Figure 1 As shown, a remote sensing multi-scale target detection network method based on explicit adjustment of target scale of the present invention is implemented in the following steps:

[0063] Follow the steps below to implement it:

[0064] Step 1: The input image I is processed by a network with several layers of structures to obtain a feature map F1;

[0065] Step 2: Generate anchor box A; specifically:

[0066] Input the preset anchor box area S = {s1, a2, ..., s K}、Preset aspect ratio parameter R={r1,r2,…,r L}; Combine the two parameters and output a set of rectangular boxes with different areas and aspect ratios at intervals of several pixels in the x and y directions A={a1,a2,…,a N}, N = K*L; output multiple groups of anchor boxes A(i,j), where i,j represent pixel positions;

[0067] Step 3: Assign explicit enlarged anchor boxes A to the proposal generation network * ;

[0068] Step 4: Explicitly enlarge the learning layer to generate G2 * ;

[0069] Step 5: Generate region proposal P;

[0070] Step 6: Assign explicit upscale proposals P to the detection network * ;

[0071] Step 7: Sampling the region of interest F P ;

[0072] Step 8: Explicitly restore the learning layer to generate G4 * ;

[0073] Step 9: The detection network predicts the target detection result D.

[0074] The invention discloses a remote sensing multi-scale target detection network based on explicit adjustment of target scale, which is essentially a convolutional neural network. The convolutional neural network includes a feature extraction network, an anchor frame generation algorithm, a region proposal network, a proposal region sampling layer, and a detection network. The improvement of the invention lies in the explicit enlargement of the anchor frame allocation layer, the explicit enlargement learning layer, the explicit enlargement of the proposal allocation layer, and the explicit restoration learning layer.

[0075] in:

[0076] (1) The input image is processed by the convolutional feature extraction network and the convolutional feature map is output; the convolutional feature extraction network can be; VGG, ResNet, ResNext, DenseNet, SqueezeNet, ShuffleNet, MobileNet

[0077] (2) The anchor frame generation algorithm generates a series of rectangular frames with different areas and aspect ratios according to the preset anchor frame size and aspect ratio parameters;

[0078] (3) Explicitly enlarge the anchor box assignment layer. During the model training phase, the generated anchor boxes are assigned to box labels whose intersection-over-union ratio is greater than a threshold, and the box labels with smaller sizes are enlarged by a specific ratio, so as to match larger anchor boxes for small objects.

[0079] (4) The region proposal network includes an anchor box region feature sampling layer, an anchor box classification layer, and an anchor box regression layer, which are used to predict region proposals: the anchor box region feature sampling layer samples the anchor box region on the above feature map to obtain the anchor box region features; the anchor box classification layer predicts whether there is a target in the anchor box based on the above anchor box region features; the anchor box regression layer predicts the position regression vector of the anchor box based on the above anchor box region features, updates the anchor box position, and obtains the region proposal.

[0080] (5) During the model training phase, the explicit magnification learning layer calculates the loss of the box labels and predicted region proposals, updates the weight parameters of the region proposal network, and magnifies the box labels with smaller label sizes by a specific ratio, thereby guiding the proposal generation network to generate larger anchor boxes for small objects;

[0081] (6) The proposal region sampling layer samples the proposal region on the feature map to obtain the region proposal features;

[0082] (7) Explicitly enlarge the proposal assignment layer. During the model training phase, the generated region proposals are assigned to the box labels whose intersection-over-union ratio is greater than a threshold, and the box labels with smaller sizes are enlarged by a specific ratio, so as to match larger proposals for small objects.

[0083] (8) The detection network includes a target classifier and a position regressor, which are used to predict the final target category and position: the target classifier predicts the specific category of the target based on the region proposal features; the position regressor predicts the position regression vector of the target based on the region proposal features, updates the proposal position, and obtains the target position.

[0084] (9) During the model training phase, the explicit restoration learning layer calculates the loss of the original-size box label and the predicted target position, updates the model weight parameters, and thus guides the detection network to predict the correct target position for small objects.

[0085] The present invention discloses a remote sensing multi-scale target detection network based on explicit adjustment of target scale, wherein the anchor frame generation algorithm is used to pre-generate a series of anchor frames of different sizes and shapes; the region proposal network is used to generate some region proposals where targets may exist based on the above convolution feature map and anchor frames; the detection network is used to predict the position and category of the target based on the above convolution feature map and region proposals.

[0086] The explicit enlarged anchor box assignment layer and the explicit enlarged learning layer are combined to act on the region proposal network during the training phase. For the region proposal network, let the input anchor box be Anchor = (x a ,y a ,w a ,h a ), explicitly enlarge the anchor box assignment layer and assign it to the box label BoxLabel = (x b ,y b ,w b ,h b ), and must meet the following requirements:

[0087]

[0088] Where θ is the preset intersection-over-union ratio threshold (usually set to 0.3). Let k be the explicit scaling factor and S be the small object scale threshold. The buffer box corresponding to the box label is:

[0089]

[0090] On the image feature map, the assigned explicitly enlarged anchor box is sampled, the anchor box features are extracted, and the proposal generation network is input to predict the target region proposal Proposal = (x p ,y p ,w p ,h p ). During model training, the region proposal network is made to learn the enlarged box label, and the loss function is:

[0091] Loss reg =SmoothL1(Proposal,BufferBox(BoxLabel,k)).

[0092] Effect analysis: First, the use of the buffer box can effectively avoid the offset sensitivity problem of small targets; second, the buffer box is used in allocating anchor boxes and learning targets. The model learns the mapping from buffer box to buffer box, and the scale difference is small, which is convenient for training a better model; finally, the buffer box can be used as an intermediate value when solving the target position. In the detection results of RPN, as long as the small target is located in this buffer box (even if it is not in the center of the buffer box), more features of the small target can be retained, thereby making it easier to predict the position of the small target and improve the recall rate.

[0093] The explicit amplification proposal allocation layer and the explicit restoration learning layer are combined to act on the detection network during the training phase. For the detection network, let the input region proposal be Proposal = (x p ,y p ,w p ,h p ), explicitly enlarge the proposal assignment layer and assign it to the box label BoxLabel = (x b ,y b ,w b ,h b ), and must meet the following requirements:

[0094]

[0095] On the image feature map, the assigned explicit enlargement proposal is sampled, the region proposal feature is extracted, input into the detection network, and the location of the target is predicted. d ,y d ,w d ,h d ). During model training, the region proposal network is made to learn the enlarged box label, and the loss function is:

[0096] Loss reg =SmoothL1(Detection,BufferBox(BoxLabel,1)).

[0097] Effect analysis: The proposal generated by the region proposal network is a buffer proposal (the predicted value corresponding to the buffer box). The proposal allocation layer is explicitly enlarged to assign the above buffer proposal to small targets to maintain the consistency of the region proposal network. The input buffer proposal has a larger receptive field than the ordinary proposal and contains more contextual features, which is helpful for auxiliary discrimination of the target category and can improve the accuracy of target detection.

[0098] The effect of the present invention is further illustrated by simulation experiments:

[0099] 1. Simulation Dataset

[0100] The public dataset HRRSD in the remote sensing field was used in the experiment.

[0101] 2. Simulation content

[0102] The method of the present invention is used to perform target detection and analysis of high-resolution remote sensing images according to the following steps:

[0103] First, ablation experiments are conducted on HRRSD, as shown in Table 1. The red bold and blue bold in the table represent the first and second place respectively. In the table, rpn_scl and det_scl represent the use of the proposed method in the region proposal network and the detection network respectively. When only rpn_scl is used, the accuracy and recall of small objects decrease. This is because the region proposal network outputs a buffer proposal, while the detection network uses a normal proposal allocation layer, which is inconsistent. When only det_scl is used, the accuracy and recall of small objects decrease. This is because the region proposal network outputs a normal proposal, while the detection network uses an explicit amplification proposal allocation layer with buffer proposals as input, which is inconsistent. When rpn_scl and det_scl are used at the same time, the output of the region proposal network and the proposal allocation layer of the detection network are consistent, so the accuracy and recall of small objects are improved by 2.02% and 1.60% respectively.

[0104] Then, the accuracy of the method of the present invention and several other methods are compared on the HRRSD database, as shown in Table 2. It can be seen that our method achieves the best detection accuracy on this database.

[0105] Table 1 Ablation experiments of the present invention in the HRRSD database

[0106]

[0107] Table 2 Comparative experiments of different methods in HRRSD database

[0108]

[0109] Example 2

[0110] like Figure 1 As shown, different from Example 1, in step 3 of the remote sensing multi-scale target detection network method based on explicit adjustment of target scale of the present invention, an explicit enlarged anchor box A is assigned to the proposal generation network. * Specifically:

[0111] Input a set of anchor boxes A = {a1, a2, ..., a n ,…,a N}, and the true target boxes (i.e., target location labels) of all targets contained in image I G = {g1, g2, …, g m ,…,g M}; Output the final target box positive sample A * ={a 1* ,a 2* ,…,a n* ,…,a N1*}, the length is changed to N1;

[0112] Enlarge the box labels with smaller size (threshold s1) in G by a specific ratio r1 to obtain G1 * = {g 1 1,g 1 2,…,g 1 m ,…,g 1 M};

[0113] The threshold s1 used for judging "small size" can be set as an absolute size (target pixel size) threshold or a relative size (ratio of target pixel size to image pixel size) threshold.

[0114] The "specific ratio r1" is optional and is generally set to 2;

[0115] Then A and G1 * Perform one-to-one matching and try to find the n Find matching g 1 m If no match is found for g 1 m Then delete a from A n ;

[0116] Here, "matching" means that the overlap evaluation index (intersection-over-union ratio) of the two frames is greater than the threshold.

[0117] Example 3

[0118] Different from Example 2, in step 4 of a remote sensing multi-scale target detection network method based on explicit adjustment of target scale of the present invention, generating a display magnification learning layer is specifically as follows:

[0119] Input the true target box (i.e. target location label) G of all targets; enlarge the box label with a smaller size (threshold s2) in G by a specific ratio r2 to obtain G2 * = {g 2 1,g 2 2,…,g 2 m ,…,g 2 M}; Output explicit enlarged learning target box G2 * ;

[0120] The threshold s2 used for judging "small size" can be set as an absolute size (target pixel size) threshold or a relative size (ratio of target pixel size to image pixel size) threshold;

[0121] The "specific ratio r2" is optional and is generally set to 2.

[0122] Example 4

[0123] Different from Example 3, in step 5 of a remote sensing multi-scale target detection network method based on explicit adjustment of target scale of the present invention, generating a region proposal P is specifically as follows:

[0124] Input feature map F1, target box positive sample pair A * , explicitly enlarge the learning target box G2 * ; Output region proposal P = {p1,p2,…,p o ,…,p O};

[0125] For each a n : Predict the target box a in the feature map F1 n The corresponding area is sampled and scaled to a specific size (usually 7*7) to obtain R;

[0126] Perform multi-layer convolution (or full connection) on R and output region proposal P = {p1, p2, ..., p o* …,p O}(including location and category);

[0127] In the training process of the model, the real target box used by the objective function of "multi-layer convolution (or full connection)" is G2 * .

[0128] Example 5

[0129] like Figure 1 As shown, different from Example 4, in step 6 of the remote sensing multi-scale target detection network method based on explicit adjustment of target scale of the present invention, an explicit amplification proposal P is allocated to the detection network. * , specifically:

[0130] Input region proposal P, true target box of all targets (i.e. target location label) G; output explicit enlarged proposal positive sample P * ={p1*,p2*,…,p o *,…,p O *};

[0131] Enlarge the box labels with smaller size (threshold s3) in G by a specific ratio r3 to obtain G3 * = {g 3 1,g 3 2,…,g 3 m ,…,g 3 M};

[0132] The threshold s3 used for judging "small size" can be set as an absolute size (target pixel size) threshold or a relative size (ratio of target pixel size to image pixel size) threshold;

[0133] The "specific ratio r3" is optional and is generally set to 2;

[0134] Then P and G3 * Perform one-to-one matching and try to find the n Find matching g 3 m If no match is found for g 3 m Then delete p from P n , get P * .

[0135] Example 6

[0136] Different from Example 5, in step 7 of the remote sensing multi-scale target detection network method based on explicit adjustment of target scale of the present invention, the region of interest sampling F P Specifically:

[0137] Input feature map F1, explicitly enlarge the proposal positive sample P * : Output feature map F of the region of interest P ={F p1* ,F p2* ,…,F po* ,…,F pO*}:

[0138] For each p o *: Predict the target box p in the feature map F1 o * The corresponding area is sampled and scaled to a specific size (usually 7*7) to obtain F po* .

[0139] Example 7

[0140] Different from Example 6, in step 8 of the remote sensing multi-scale target detection network method based on explicit adjustment of target scale of the present invention, the explicit restoration learning layer generates G4 * Specifically:

[0141] Input the true target box (i.e. target location label) G of all targets; enlarge the box label with smaller size (threshold s4) in G by a specific ratio r4 to obtain G4 * = {g 2 1,g 2 2,…,g 2 m ,…,g 2 M}; Output explicit restoration learning target frame G4 * ;

[0142] The threshold s4 used for judging "small size" can be set as an absolute size (target pixel size) threshold or a relative size (ratio of target pixel size to image pixel size) threshold;

[0143] The "specific ratio r4" is optional and is generally set to 1.

[0144] Example 8

[0145] Different from Example 5, in step 9 of a remote sensing multi-scale target detection network method based on explicit adjustment of target scale of the present invention, the detection network predicts the target detection result D as follows:

[0146] Input the feature map F of the region of interest P , explicitly enlarge the learning target box G4 * ; for F P Perform multi-layer convolution (or full connection); output detection result D = {d1, d2, ..., d h* …,d H}(including location and category);

[0147] In the training process of the model, the real target box used by the objective function of "multi-layer convolution (or full connection)" is G4 * .

[0148] The above description shows and describes several preferred embodiments of the invention, but as mentioned above, it should be understood that the invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the invention concept described herein through the above teachings or the technology or knowledge of the relevant field. Changes and modifications made by those skilled in the art should be within the scope of protection of the claims attached to the invention without departing from the spirit and scope of the invention.

Claims

1. A remote sensing multi-scale target detection network method based on explicit adjustment of target scale, characterized in that: Follow the steps below to implement it: Step 1, calculate the feature map F1 of the input image I; Step 2: Generate anchor box A; Step 3: Assign explicit enlarged anchor boxes A to the proposal generation network * ; Step 4: Explicitly enlarge the learning layer to generate G2 * ; Step 5: Generate region proposal P; Step 6: Assign explicit upscale proposals P to the detection network * ; Step 7: Sampling the region of interest F P ; Step 8: Explicitly restore the learning layer to generate G4 * ; Step 9: The detection network predicts the target detection result D.

2. The remote sensing multi-scale target detection network method based on explicit adjustment of target scale according to claim 1, characterized in that: In step 1, the input image I is processed by a network with several layers of structure to obtain a feature map F1.

3. The remote sensing multi-scale target detection network method based on explicit adjustment of target scale according to claim 2 is characterized in that: In step 2, generating anchor frame A is specifically as follows: Input the preset anchor box area S = {s1, a2, ..., s K }、Preset aspect ratio parameter R={r1,r2,…,r L }; Combine the two parameters and output a set of rectangular boxes with different areas and aspect ratios at intervals of several pixels in the x and y directions A={a1,a2,…,a N }, N = K*L; output multiple groups of anchor boxes A(i,j), where i,j represent pixel positions.

4. The remote sensing multi-scale target detection network method based on explicit adjustment of target scale according to claim 3 is characterized in that: In step 3, the proposal generation network is assigned an explicitly enlarged anchor box A * Specifically: Input a set of anchor boxes A = {a1, a2, ..., a n ,…,a N }, and the true target boxes G = {g1,g2,…,g m ,…,g M }; Output the final target box positive sample A * ={a 1* ,a 2* ,…,a n* ,…,a N1* }, the length is changed to N1; Enlarge the smaller box labels in G by a specific ratio r1 to obtain G1 * = {g 1 1,g 1 2,…,g 1 m ,…,g 1 M }; The threshold s1 used for judging "small size" can be set as an absolute size threshold or a relative size threshold; The "specific ratio r1" is set to 2; Then A and G1 * Perform one-to-one matching and try to find the n Find matching g 1 m If no match is found for g 1 m Then delete a from A n ; Here, "match" means that the overlap evaluation index of the two frames is greater than the threshold.

5. The remote sensing multi-scale target detection network method based on explicit adjustment of target scale according to claim 4 is characterized in that: In step 4, generating the display magnification learning layer is specifically as follows: Input the true target box G of all targets; enlarge the smaller box labels in G by a specific ratio r2 to obtain G2 * = {g 2 1,g 2 2,…,g 2 m ,…,g 2 M }; Output explicit enlarged learning target box G2 * ; The threshold s2 used for judging "small size" can be set as an absolute size threshold or a relative size threshold; The "specific ratio r2" is set to 2.

6. The remote sensing multi-scale target detection network method based on explicit adjustment of target scale according to claim 5, characterized in that: In step 5, generating the region proposal P is specifically as follows: Input feature map F1, target box positive sample pair A * , explicitly enlarge the learning target box G2 * ; Output region proposal P = {p1,p2,…,p o ,…,p O }; For each a n : Predict the target box a in the feature map F1 n The corresponding area is sampled and scaled to the set size to obtain R; Perform multi-layer convolution or full connection on R, and output region proposal P = {p1, p2, ..., p o* …,p O }; In the training process of the model, the real target box used by the objective function of "multi-layer convolution or full connection" is G2 * .

7. The remote sensing multi-scale target detection network method based on explicit adjustment of target scale according to claim 6, characterized in that: In step 6, the detection network is assigned an explicit amplification proposal P * , specifically: Input region proposal P, true target box G of all targets; output explicit enlarged proposal positive sample P * ={p1*,p2*,…,p o *,…,p O *}; Enlarge the smaller box labels in G by a specific ratio r3 to obtain G3 * = {g 3 1,g 3 2,…,g 3 m ,…,g 3 M }; The threshold s3 used for judging "small size" can be set as an absolute size threshold or a relative size threshold; The "specific ratio r3" is set to 2; Then P and G3 * Perform one-to-one matching and try to find the n Find matching g 3 m If no match is found for g 3 m Then delete p from P n , get P * .

8. The remote sensing multi-scale target detection network method based on explicit adjustment of target scale according to claim 7, characterized in that: In step 7, the region of interest is sampled F P Specifically: Input feature map F1, explicitly enlarge the proposal positive sample P * : Output feature map F of the region of interest P ={F p1* ,F p2* ,…,F po* ,…,F pO* }: For each p o *: Predict the target box p in the feature map F1 o * The corresponding area is sampled and scaled to the set size to obtain F po* .

9. The remote sensing multi-scale target detection network method based on explicit adjustment of target scale according to claim 8, characterized in that: In step 8, the explicit recovery learning layer generates G4 * Specifically: Input the true target box G of all targets; enlarge the smaller box labels in G by a specific ratio r4 to obtain G4 * = {g 2 1,g 2 2,…,g 2 m ,…,g 2 M }; Output explicit restoration learning target frame G4 * ; The threshold s4 used for judging "small size" can be set as an absolute size threshold or a relative size threshold; The "specific ratio r4" is set to 1.

10. The remote sensing multi-scale target detection network method based on explicit adjustment of target scale according to claim 9, characterized in that: In step 9, the detection network predicts the target detection result D as follows: Input the feature map F of the region of interest P , explicitly enlarge the learning target box G4 * ; for F P Perform multi-layer convolution or full connection; output detection result D = {d1, d2, ..., d h* …,d H }; In the training process of the model, the real target box used by the objective function of "multi-layer convolution or full connection" is G4 * .

Images