A weakly supervised aircraft detection method for remote sensing images based on key points
Aircraft detection in remote sensing images is enhanced through weak supervision using key points to generate segmentation masks, reducing annotation costs and improving accuracy and robustness against noise and multi-resolution images.
Patent Information
- Application Number
- CN202111562488.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-20
- Publication Date
- 2024-12-24
- Estimated Expiration
- 2041-12-20
AI Technical Summary
Existing aircraft detection methods in remote sensing images face challenges due to the high cost of segmentation mask annotations, making it difficult to use advanced detection models like Mask RCNN, and are not robust against noise and multi-resolution images.
Aircraft detection based on weak supervision using key points, where key point annotations are used to generate segmentation masks, reducing annotation workload and improving detection accuracy through a method that includes constructing aircraft segmentation masks, cutting images into uniform sizes, and using a detection model that predicts both detection boxes and segmentation masks.
This method reduces annotation costs, enhances detection accuracy, improves F1 value, and provides robustness against noise and multi-resolution images, while maintaining generalization ability.
Smart Images
Figure CN114266971B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a remote sensing image aircraft detection method, in particular to a key point-based weakly supervised remote sensing image aircraft detection method. Background Art
[0002] Accurately detecting and identifying aircraft at airports through remote sensing images plays an important role in real-time monitoring and management of airports. In recent years, deep learning technology has developed rapidly in the field of computer vision, and target detection is one of the most important tasks. It has achieved extremely high accuracy, recall rate and F1 value (F1 score is: ), so deep learning algorithms are often used to detect targets such as aircraft in remote sensing images.
[0003] At present, the target detection algorithms based on deep learning are mainly divided into two-stage target detection algorithms represented by the Region-Convolutional Neural Network (R-CNN) series, such as FasterRCNN and Mask RCNN, and single-stage target detection algorithms represented by the Single Shot Multibox Detector (SSD) and the Single Shot You Only Look Once (YOLO) series. The two-stage detection algorithm first generates a large number of candidate regions through the region generation network RPN, and then sends the candidate regions to the neural network to predict the location information and category probability of its target. Its target detection accuracy is high. Therefore, the two-stage model usually has better detection effect in remote sensing images with complex backgrounds and large frames. On the other hand, with the introduction of Mask RCNN, the method of using detection box annotation and instance segmentation mask annotation in the training process has been adopted by various detection models, such as Cascade Mask RCNN, Hybrid Task RCNN, etc., and has been proven to significantly improve the accuracy of target detection. However, since segmentation mask annotation is extremely expensive, most self-constructed datasets do not contain segmentation mask annotations, making such methods difficult to use. Summary of the invention
[0004] Purpose of the invention: The technical problem to be solved by the present invention is to provide a method for aircraft detection in remote sensing images based on weak supervision of key points in view of the shortcomings of the prior art.
[0005] In order to solve the above technical problems, the present invention discloses a method for aircraft detection in remote sensing images based on weak supervision of key points.
[0006] A method for detecting aircraft in remote sensing images based on weak supervision of key points includes the following steps:
[0007] Step 1: construct an aircraft segmentation mask annotation based on the coordinate annotation of the aircraft key points on the original remote sensing image;
[0008] Step 2, cutting the original wide-format remote sensing image into small-format remote sensing images of uniform size to form a small-format remote sensing image set, and mapping the coordinates marked by the aircraft segmentation mask in step 1 to the small-format remote sensing images of uniform size;
[0009] Step 3, using a remote sensing image aircraft detection model based on detection box annotation and segmentation mask annotation, and training the remote sensing image aircraft detection model based on the above detection box annotation and segmentation mask annotation;
[0010] Step 4: Cut the remote sensing image to be detected into a small remote sensing image set and input it into the remote sensing image aircraft detection model trained in step 3 to obtain the aircraft detection result, i.e., the detection frame coordinates. The detection frame coordinates are restored to the original remote sensing image. Based on the confidence level, the non-maximum suppression method is used to filter the detection frames of repeated detection to obtain the final detection frame coordinate information.
[0011] Step 1 in the present invention comprises:
[0012] Step 1-1, the aircraft key point coordinates include: aircraft head key point coordinates (x1, y1), right wing end key point coordinates (x2, y2), aircraft tail key point coordinates (x3, y3), left wing end key point coordinates (x4, y4) and wing connection key point coordinates (x5, y5); connect the aircraft head key point and the tail key point to generate a head and tail line segment, connect the left wing end key point and the wing connection key point to generate a left wing line segment, connect the right wing end key point and the wing connection key point to generate a right wing line segment;
[0013] Step 1-2, expanding the head and tail line segments, the left wing line segment, and the right wing line segment in step 1-1 into rectangles, corresponding to the fuselage including the tail part, the left wing part, and the right wing part, respectively, and the expansion direction is perpendicular to the above line segments;
[0014] Step 1-3, merge the fuselage including the tail part, the left wing part and the right wing part, generate the aircraft segmentation mask, extract the key points of the boundary contour of the segmentation mask, and obtain the coordinates of the key points of the boundary contour of the segmentation mask.
[0015] In step 2 of the present invention, the original remote sensing image is cut into small images of uniform size in a sliding frame manner, the length x and width y of the small images are equal, and the length of the overlapping area is z.
[0016] In step 1-3 of the present invention, the coordinates of the key points of the boundary contour of the aircraft segmentation mask are mapped to a small image as training annotations for the remote sensing image aircraft detection model.
[0017] In step 3 of the present invention, a target detection model based on detection box annotation and segmentation mask annotation is adopted. During the inference process, the model simultaneously predicts the detection box coordinates of the target and the segmentation mask of the target.
[0018] During the training process described in step 3 of the present invention, the remote sensing image aircraft detection model simultaneously predicts the detection frame position coordinates and segmentation results of all aircraft targets in the image, calculates the coordinate loss function and the segmentation loss function respectively, adds them together, and then performs back propagation to update the model parameters.
[0019] Step 4 of the present invention comprises:
[0020] Step 4-1, cutting the image to be detected into a set of small images in a sliding frame manner, normalizing the size, and obtaining a normalized image;
[0021] Step 4-2, sending the normalized images to the remote sensing image aircraft detection model in sequence to obtain aircraft detection results, namely, detection frame coordinates and segmentation masks;
[0022] Step 4-3, restore the coordinates of the aircraft detection frame to the original remote sensing image, and filter the detection frames of repeated detection based on the confidence level by using the non-maximum suppression method. That is, when the overlapping area of the detection frames exceeds the threshold, the one with the highest confidence level is retained to obtain the final aircraft detection result.
[0023] In step 4-3 of the present invention, the coordinates (x, y, w, h) of the aircraft detection result are restored to the original remote sensing image coordinates, where (x i ,y i ) is the coordinate of the starting point of the ith small image in the original remote sensing image, (x, y) is the coordinate of the aircraft detection frame in the small image, (w, h) is the length and width of the detection frame, (x ′ ,y ′ ) is the coordinate of the aircraft detection frame in the original image after restoration;
[0024] (x ′ ,y ′ ,w,h)=(x+x i ,y+y i ,w,h).
[0025] In step 4-3 of the present invention, the detection frames restored by coordinates are sorted from high to low according to the confidence level to form a detection frame list, and the detection frame list is screened. The method for screening the detection frame list is: select the detection frame box0 with the highest confidence level, calculate the intersection of its area with the area covered by the remaining detection frames i, and the coverage area percentage Iou i ,
[0026]
[0027] When the proportion is greater than the preset threshold θ, the detection box is removed from the detection box list, and the detection boxes in the detection box list are screened in turn to obtain the final detection result.
[0028] Beneficial effects:
[0029] (1) The invention proposes a method for aircraft detection in remote sensing images based on key points and weak supervision. The method uses key points to generate segmentation mask annotations. Compared with the detection model based on detection box annotation and segmentation mask annotation, this method is weakly supervised learning, which greatly reduces the annotation workload. Compared with the detection model based on single detection box annotation, this method improves the F1 value of multi-scale aircraft target detection. At the same time, during the inference process, the model can provide segmentation masks for detecting aircraft.
[0030] (2) This method is universal. In other image and video rigid object detection tasks, this method can be used for object detection. When the annotation is limited, the best existing (State of The Art) detection model architecture based on detection box annotation and segmentation mask annotation can be selected and a pre-trained model can be used. Compared with the detection model based on single detection box annotation, the detection model based on detection box annotation and segmentation mask annotation usually has better detection accuracy, and the accuracy will be further improved after using the pre-trained model.
[0031] (3) The key point-based weakly supervised remote sensing image aircraft detection method proposed in this invention has strong generalization ability and can process multi-resolution remote sensing images;
[0032] (4) The key point-based weakly supervised remote sensing image aircraft detection method proposed in the present invention has improved robustness and can handle noise interference such as cloud occlusion and low resolution in remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.
[0034] Figure 1 This is a schematic diagram of the workflow framework.
[0035] Figure 2 The figure is a schematic diagram of the specific work process.
[0036] Figure 3 Generate schematic diagram for aircraft segmentation mask annotation.
[0037] Figure 4This is a schematic diagram of the sliding frame cutting.
[0038] Figure 5 Schematic diagram of the target detection model architecture.
[0039] Figure 6 Diagram for filtering the test results. DETAILED DESCRIPTION
[0040] First, the segmentation mask annotation of the aircraft is constructed based on the key points, and then the original remote sensing image is cut into small images, and the detection box annotation and the segmentation mask annotation are mapped to the small image; the aircraft annotation box and the segmentation mask annotation are combined to train the target detection model; in the testing phase, the wide image is cut into small images in the same way, and the small images are sent to the trained detection model in turn to obtain the detection results; the detection box coordinates are restored to the original image, and the repeated detection results are filtered out according to the detection box confidence and the size of the overlapping area through non-maximum suppression to form the final detection result.
[0041] The present invention provides a method for detecting aircraft in remote sensing images based on weak supervision of key points, comprising the following steps:
[0042] Step 1: Construct aircraft segmentation mask annotation based on aircraft key point annotation;
[0043] Step 2, cutting the original wide-format remote sensing image into small-format remote sensing images of uniform size to form a small-format image set, and mapping the aircraft segmentation mask annotation coordinates to the small-format remote sensing images of uniform size;
[0044] Step 3, using a remote sensing image aircraft detection model based on detection box annotation and segmentation mask annotation, based on the above detection box annotation and segmentation mask annotation, training the remote sensing image aircraft detection model segmentation mask annotation;
[0045] Step 4: Cut the remote sensing image to be detected into a small remote sensing image set and input it into the remote sensing image aircraft detection model trained in step 3 to obtain the aircraft detection result, i.e., the detection frame coordinates. The detection frame coordinates are restored to the original remote sensing image. Based on the confidence level, the non-maximum suppression method is used to filter the detection frames of repeated detection to obtain the final detection frame coordinate information.
[0046] Furthermore, in one implementation, step 1 includes:
[0047] Step 1-1, the key points include: aircraft head (x1, y1), right wing end (x2, y2), aircraft tail (x3, y3), left wing end (x4, y4) and wing connection (x5, y5); connect the key points of the aircraft head and tail to generate line segment 1, connect the key points of the left wing end and the wing connection to generate line segment 2, connect the key points of the right wing end and the wing connection to generate line segment 3;
[0048] Step 1-2, expand line segment 1, line segment 2, and line segment 3 into rectangles, corresponding to the fuselage part (including the tail), the left wing part, and the right wing part, respectively. The expansion direction is perpendicular to the line segment, and the width of the rectangle is determined by the data set;
[0049] Step 1-3, merge the fuselage, left wing and right wing to generate an aircraft segmentation template, and extract the key points of the segmentation template boundary contour (a0, b0, ………, a i ,b i ,………) as segmentation mask annotation.
[0050] Furthermore, in one implementation, step 2 includes:
[0051] Step 2-1, since the original remote sensing image is usually large, usually more than 10000 x 10000, it is difficult to match the existing detection model architecture, so the original remote sensing image is cut into a set of small cut images of uniform size in a sliding frame manner, and each small image has equal length and width (x = y). The size of the input image of the commonly used remote sensing image target detection model is about 512 x 512 or 1000 x 1000. Since the size of the small image is fixed, some targets will be cut into two parts, so it is set that two adjacent small cut images have a certain overlap area, and the length of the overlap area is z, to ensure that the complete target will appear in at least one small cut image.
[0052] Step 2-2, map the coordinates of the aircraft detection box annotation and the segmentation mask annotation to a small image as the object detection model training annotation.
[0053] Furthermore, in one implementation, step 3 includes:
[0054] Step 3-1, using the existing target detection model based on detection box annotation and segmentation mask annotation, the model simultaneously outputs the detection box coordinates of the predicted target and the segmentation mask of the target during the inference process;
[0055] Step 3-2: During the training process, the model simultaneously predicts the coordinates of the detection boxes and segmentation results of all aircraft targets in the image. The detection box position coordinate prediction is a regression problem, using the L1 or L2 loss function. The segmentation prediction is a classification problem, using the cross entropy loss function. The coordinate loss function and the segmentation loss function are calculated separately and then added together, and then back propagation is performed to update the model parameters.
[0056] Furthermore, in one implementation, step 4 includes:
[0057] Step 4-1, cut the test remote sensing image into small images according to the above sliding frame method, the length and width of the small images are equal and the same as those in training, and obtain a normalized image;
[0058] Step 4-2, sending the normalized images to the remote sensing image target detection model in sequence to obtain the aircraft detection results including the detection frame coordinates and the segmentation mask;
[0059] Step 4-3, restore the coordinates of the aircraft detection result to the original image, and filter the detection frames of repeated detections based on the confidence level by using the non-maximum suppression method, that is, when the overlapping area of the detection frames exceeds the threshold, retain the one with the highest confidence level to obtain the final aircraft detection result;
[0060] Furthermore, in one implementation, step 4-3 includes:
[0061] Step 4-3-1, restore the aircraft detection results (x, y, w, h) to the original remote sensing image coordinates, as shown below, (x i ,y i ) is the coordinate of the starting point of the ith small image in the original remote sensing image.
[0062] (x ′ ,y ′ ,w,h)=(x+x i ,y+y i ,w,h)
[0063] Step 4-3-2: Sort the detection frames restored by coordinates from high to low according to the confidence level to form a detection frame list. First, select the detection frame box0 with the highest confidence level in the list and calculate the intersection of its area with the remaining detection frames. i , and coverage area ratio IoU i ,
[0064]
[0065] If the proportion is greater than the set threshold θ, the detection frame is removed from the detection frame list. Similarly, the detection frames in the list are filtered and screened to obtain the detection result, which is the final detection result of the remote sensing image.
[0066] Example
[0067] The embodiment discloses a key point-based weakly supervised remote sensing image aircraft detection method, which is applied to multi-scale aircraft target detection tasks in remote sensing images.
[0068] like Figure 1 As shown, a schematic diagram of the workflow framework of a method for detecting aircraft in remote sensing images based on weak supervision of key points provided in this embodiment. A method for detecting aircraft in remote sensing images based on weak supervision of key points provided in this embodiment comprises the following steps:
[0069] Step 1: Construct aircraft segmentation mask annotation based on aircraft key point annotation;
[0070] Step 2, cutting the original wide-format remote sensing image into small-format remote sensing images of uniform size to form a small-format image set, and mapping the aircraft segmentation mask annotation coordinates to the small-format remote sensing images of uniform size;
[0071] Step 3, constructing a remote sensing image aircraft detection model based on detection box annotation and segmentation mask annotation, and training the remote sensing image aircraft detection model segmentation mask annotation based on the above detection box annotation and segmentation mask annotation;
[0072] Step 4: Cut the remote sensing image to be detected into a small remote sensing image set and input it into the remote sensing image aircraft detection model trained in step 3 to obtain the aircraft detection result, i.e., the detection frame coordinates. The detection frame coordinates are restored to the original remote sensing image. Based on the confidence level, the non-maximum suppression method is used to filter the detection frames of repeated detection to obtain the final detection frame coordinate information.
[0073] Figure 2 A specific workflow diagram of a method for aircraft detection in remote sensing images based on key points and weak supervision is provided in this embodiment; wherein each specific workflow diagram is similar to Figure 1 The workflow framework in Figure 1 One-to-one correspondence, that is, corresponding to the processing of the original image through steps 1 to 4 in this method.
[0074] In the method for detecting aircraft in remote sensing images based on weak supervision of key points described in this embodiment, step 1 includes:
[0075] Step 1-1, such as Figure 3 As shown in (a), the key points include: the aircraft head (x1, y1), the right wing end (x2, y2), the aircraft tail (x3, y3), the left wing end (x4, y4) and the wing connection (x5, y5); Figure 3 As shown in (b), connect the key points of the aircraft head and tail to generate line segment 1, connect the key points of the left wing end and the wing connection to generate line segment 2, and connect the key points of the right wing end and the wing connection to generate line segment 3;
[0076] Step 1-2, such as Figure 3 As shown in (c), line segment 1, line segment 2, and line segment 3 are expanded into rectangles, corresponding to the fuselage part (including the tail), the left wing part, and the right wing part, respectively. The expansion direction is perpendicular to the line segment. The width of the rectangle is determined by the data set. In this example, the width of the rectangular fuselage is selected as 7, and the width of the wing rectangle is selected as 5.
[0077] Steps 1-3, such as Figure 3As shown in (d) and (e), the fuselage, left wing, and right wing are combined to generate an aircraft segmentation template, and the key points of the segmentation template boundary contour (a0, b0, ………, a i ,b i ,………) as the model training annotation. In this example, the training set annotation adopts the Microsoft COCO (MSCOCO) format. The annotation includes the target category, detection box (upper left point coordinates, length, width), and segmentation mask contour key points.
[0078] In the method for detecting aircraft in remote sensing images based on weak supervision of key points described in this embodiment, step 2 includes:
[0079] Step 2-1, since the original remote sensing image is usually large in size and difficult to match the existing detection model architecture, the original remote sensing image is cut into a set of small cut images of uniform size in a sliding frame manner, and each small image has equal length and width (x=y). Since the size of the small image is fixed, some targets will be cut into two parts, so it is set that two adjacent small cut images have a certain overlap area, and the length of the overlap area is z, to ensure that the complete target will appear in at least one small cut image. Figure 4 As shown, in this example, the size of the small cut image is x=1024, y=1024, and the length of the overlapping area is set to z=256.
[0080] Step 2-2, map the aircraft detection box annotation and the segmentation mask annotation to a small image as the object detection model training annotation.
[0081] In the method for detecting aircraft in remote sensing images based on weak supervision of key points described in this embodiment, step 3 includes:
[0082] Step 3-1, such as Figure 5 As shown in the figure, a remote sensing image aircraft detection model based on detection box annotation and segmentation mask annotation is constructed. In this experiment, FasterRCNN is selected as the basic detection model (based on single detection box annotation). After adding the target segmentation module, it becomes MaskRCNN (based on detection box annotation and segmentation mask annotation). The basic feature extraction network is ResNet50.
[0083] Step 3-2: During the training process, the model simultaneously predicts the position coordinates of all target detection boxes and the target segmentation results. The prediction of the detection box position coordinates is a regression problem, using the L1 or L2 loss function, and the segmentation prediction is a classification problem, using the cross entropy loss function CrossEntropyLoss (Loss ce ), as shown below.
[0084] L1=|p(I i )-yi |
[0085] L2=|p(I i )-y i | 2
[0086]
[0087] Where p(I i ) is the detection result output of the i-th image, y i is the annotation of the i-th image, p(I ij ) is the predicted output of the classification of the jth pixel in the ith image, y ij is the category label of the jth pixel in the i-th image.
[0088] In the method for detecting aircraft in remote sensing images based on weak supervision of key points described in this embodiment, step 4 includes:
[0089] Step 4-1, cut the test remote sensing image into small images according to the above sliding frame method. The small image is a normalized image with the same length and width as that in training;
[0090] Step 4-2, sending the normalized images to the remote sensing image target detection model in sequence to obtain the aircraft detection result;
[0091] Step 4-3, restore the coordinates of the aircraft detection result to the original image, filter the duplicate detection frames based on the confidence of the aircraft detection frames, and when the overlapping area of the detection frames exceeds a threshold, retain the one with the highest confidence to obtain the final aircraft detection result;
[0092] Furthermore, in one implementation, step 4-3 includes:
[0093] Step 4-3-1, restore the aircraft detection results (x, y, w, h) to the original remote sensing image coordinates, as shown below, (x i ,y i ) is the coordinate of the starting point of the ith small image in the original remote sensing image.
[0094] (x ′ ,y ′ ,w,h)=(x+x i ,y+y i ,w,h)
[0095] Step 4-3-2: Sort the detection frames restored by coordinates from high to low according to the confidence level to form a detection frame list. First, select the detection frame box0 with the highest confidence level in the list and calculate the intersection of its area with the remaining detection frames. i , and coverage area ratio IoUi ,
[0096]
[0097] If the proportion is greater than the set threshold θ, the detection box is removed from the detection box list. Similarly, the detection boxes in the list are filtered to obtain the final detection result, which is the final detection result. Figure 6 As shown in the example, due to sliding window detection, detection frames 1, 2, and 3 are obtained for the same target, which are sorted into 2, 1, and 3 according to the confidence level. The IoU1 and IoU2 of detection frames 2 and 1 and 3 are calculated respectively. In the experiment, we set the threshold θ to 0.1, so detection frames 1 and 3 are filtered out and detection frame 2 is retained.
[0098] In a specific implementation, the present invention further provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment of the remote sensing image fine-grained aircraft recognition method based on local segmentation and feature fusion provided by the present invention. The storage medium may be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.
[0099] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention or some parts of the embodiments.
[0100] The present invention provides a method and idea for a method for aircraft detection in remote sensing images based on weak supervision of key points. There are many methods and approaches to implement the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be considered as the scope of protection of the present invention. All components not specified in this embodiment can be implemented using existing technologies.
Claims
1. A method for aircraft detection in remote sensing images based on weak supervision of key points, characterized in that: The following steps are involved: Step 1: construct an aircraft segmentation mask annotation based on the coordinate annotation of the aircraft key points on the original remote sensing image; Step 2, cutting the original wide-format remote sensing image into small-format remote sensing images of uniform size to form a small-format image set, and mapping the coordinates marked by the aircraft segmentation mask in step 1 to the small-format remote sensing images of uniform size; Step 3, using a remote sensing image aircraft detection model based on detection box annotation and segmentation mask annotation, and training the remote sensing image aircraft detection model based on the above detection box annotation and segmentation mask annotation; Step 4, cutting the remote sensing image to be detected into a small remote sensing image set and inputting the remote sensing image aircraft detection model trained in step 3 to obtain the aircraft detection result, i.e., the detection frame coordinates and the segmentation mask, and restoring the detection frame coordinates to the original remote sensing image. Based on the confidence, the non-maximum suppression method is used to filter the detection frames of repeated detection to obtain the final detection frame coordinate information; Wherein, step 1 comprises: Step 1-1, the aircraft key point coordinates include: aircraft head key point coordinates (x1, y1), right wing end key point coordinates (x2, y2), aircraft tail key point coordinates (x3, y3), left wing end key point coordinates (x4, y4) and wing connection key point coordinates (x5, y5); connect the aircraft head key point and the tail key point to generate a head and tail line segment, connect the left wing end key point and the wing connection key point to generate a left wing line segment, connect the right wing end key point and the wing connection key point to generate a right wing line segment; Step 1-2, expanding the head and tail line segments, the left wing line segment, and the right wing line segment in step 1-1 into rectangles, corresponding to the fuselage including the tail part, the left wing part, and the right wing part, respectively, and the expansion direction is perpendicular to the above line segments; Step 1-3, merge the fuselage including the tail part, the left wing part and the right wing part, generate the aircraft segmentation mask, extract the key points of the boundary contour of the segmentation mask, and obtain the coordinates of the key points of the boundary contour of the segmentation mask.
2. According to the method of weakly supervised remote sensing image aircraft detection based on key points in claim 1, it is characterized in that: In step 2, the original remote sensing image is cut into small images of uniform size in a sliding frame manner. The length x and width y of the small image are equal, and the length of the overlapping area is z.
3. The method for detecting aircraft in remote sensing images based on weak supervision of key points according to claim 2 is characterized in that: The coordinates of the key points of the boundary contour of the aircraft segmentation mask in steps 1-3 are mapped to a small image as training annotations for the remote sensing image aircraft detection model.
4. The method for detecting aircraft in remote sensing images based on weak supervision of key points according to claim 3 is characterized in that: In step 3, a target detection model based on detection box annotation and segmentation mask annotation is used. During the inference process, the model simultaneously predicts the detection box coordinates of the target and the segmentation mask of the target.
5. The method for aircraft detection in remote sensing images based on weak supervision of key points according to claim 4 is characterized in that: During the training process described in step 3, the remote sensing image aircraft detection model simultaneously predicts the detection box position coordinates and segmentation results of all aircraft targets in the image, calculates the coordinate loss function and the segmentation loss function respectively, adds them together, and then performs back propagation to update the model parameters.
6. The method for aircraft detection in remote sensing images based on weak supervision of key points according to claim 5 is characterized in that: Step 4 includes: Step 4-1, cut the image to be detected into a set of small images in a sliding frame manner, the size of which is the same as that during training, and obtain a normalized image; Step 4-2, sending the normalized images to the remote sensing image aircraft detection model in sequence to obtain aircraft detection results, namely, detection frame coordinates and segmentation masks; Step 4-3, restore the coordinates of the aircraft detection frame to the original remote sensing image, and filter the detection frames of repeated detection based on the confidence level by using the non-maximum suppression method. That is, when the overlapping area of the detection frames exceeds the threshold, the one with the highest confidence level is retained to obtain the final aircraft detection result.
7. The method for aircraft detection in remote sensing images based on weak supervision of key points according to claim 6 is characterized in that: In step 4-3, the coordinates (x, y, w, h) of the aircraft detection result are restored to the original remote sensing image coordinates, where (x i ,y i ) is the coordinate of the starting point of the ith small image in the original remote sensing image, (x, y) is the coordinate of the aircraft detection frame in the small image, (w, h) is the length and width of the detection frame, and (x′, y′) is the coordinate of the aircraft detection frame in the original image after restoration; (x′,y′,w,h)=(x+x i ,y+y i ,w,h)。 8. The method for aircraft detection in remote sensing images based on weak supervision of key points according to claim 7 is characterized in that: In step 4-3, the detection boxes restored by coordinates are sorted from high to low according to the confidence level to form a detection box list, and the detection box list is filtered.
9. The method for aircraft detection in remote sensing images based on weak supervision of key points according to claim 8 is characterized in that: In step 4-3, the method for filtering the detection box list is: select the detection box box0 with the highest confidence, and calculate the intersection of its area with the area covered by the remaining detection boxes. i , and coverage area ratio IoU i , When the proportion is greater than the preset threshold θ, the detection box is removed from the detection box list, and the detection boxes in the detection box list are screened in turn to obtain the final detection result.