A method for identifying floating objects on the water surface from UAV aerial photography based on neural network

By improving the detection method of floating objects on the surface of drone aerial photography, the ResNet-50 network, CC-FPN network and S-RPN network are used, combined with the ROI Align layer and bilinear interpolation method, the shortcomings in drone detection accuracy and speed are solved, especially in complex environments, the detection effect of small targets has been significantly improved.

CN115512248BActive Publication Date: 2025-08-22郑珂
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202211197896.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-29
Publication Date
2025-08-22
Estimated Expiration
2042-09-29

AI Technical Summary

Technical Problem

The existing drone aerial photography of floating objects on the water surface has insufficient detection accuracy and speed, especially in dealing with small target detection in complex environments.

Method used

The ResNet-50 network is used to replace the VGG-16 network, combine the CC-FPN network and the S-RPN network, and use the ROI Align layer instead of the ROI Pooling layer, and the feature map is fused through the CAG module, and the detection accuracy is improved by using a bilinear interpolation method.

Benefits of technology

It improves the detection speed and accuracy of aerial images of drones, especially the ability to identify small targets in complex environments, and improves the accuracy of classification of floating objects on the water surface.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115512248B_ABST
    Figure CN115512248B_ABST
Patent Text Reader

Abstract

A neural network-based method for identifying floating objects on the water surface in aerial photography taken by unmanned aerial vehicles (UAVs) comprises the following steps: inputting an aerial image of floating objects on the water surface into a ResNet-50 network to obtain a feature map, inputting the feature map into a CC-FPN network to generate a fused feature map, inputting the fused feature map into an S-RPN network to generate a candidate frame, inputting the fused feature map and the candidate frame into a ROIAlign layer to generate a candidate feature map, and inputting the candidate feature map into a fully connected layer to generate a detection frame. The detection method uses the more expressive ResNet-50 network and the CC-FPN network using a CAG module, as well as the more accurate S-RPN network. The ROIAlign layer with better matching performance is used as the region of interest layer, thereby achieving better detection results for UAV aerial images and more accurate classification of floating objects on the water surface.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence neural network technology, and in particular to a method for identifying floating objects on the water surface in aerial photography by an unmanned aerial vehicle (UAV) based on a neural network. Background Art

[0002] The rapid development of artificial intelligence has placed higher demands on efficient intelligent vision systems. To address more complex semantic tasks in computer vision, such as object recognition, behavior analysis, and motion analysis, researchers have developed numerous detection and tracking algorithms over the past few decades. 2012 is considered a milestone in object detection. Before 2012, most object detection methods were traditional, such as HOG (Histogram of Oriented Gradient) and DPM (Deformable Part Model). Since 2012, with the development of deep learning, object detection algorithms based on convolutional neural networks can be divided into two categories: one is regression-based algorithms, including SSD (Single Shot MultiBox Detector), YOLO (You Only Look Once), and Retina-Net; the other is region-based algorithms, including R-CNN (Region-based Convolutional Neural Networks), Fast R-CNN, Faster R-CNN, SPPNet (Spatial Pyramid Pooling Convolutional Networks), and R-FCN (Region-based Fully Convolutional Network). The former has advantages in detection speed, while the latter has advantages in detection precision and accuracy. Object detection technology is currently widely used in industries such as industrial defect detection, vehicle visual tracking, and medical image analysis.

[0003] With the advancement of science and technology, human production and life have undergone tremendous changes. At the same time, the environmental problems brought about by social development are becoming increasingly serious. Among them, water pollution has always been a high priority. As a global issue, surface water pollution and sub-surface water pollution are particularly serious. Among them, the problem of floating debris in surface water pollution has gradually come into people's attention. Due to the presence of floating debris such as plants, weeds, plastic products, and metal products, water pollution and ecological damage are particularly serious. Benefiting from the booming drone industry, UAVs (Unmanned Aerial Vehicles) have been applied to many fields such as security monitoring, search and rescue, and motion analysis. Unlike traditional surveillance cameras, drone mobile cameras have the advantages of easy deployment, high maneuverability, a wide field of view, and uniform scale. Currently, drone application scenarios are becoming more and more extensive, such as drone aerial refueling and drone tracking.

[0004] Floating objects on the water surface are increasingly polluting the aquatic environment, and long-term accumulation will lead to further deterioration of the water environment. Therefore, the detection of floating objects on the water surface is of great significance to water purification. In order to make up for the shortcomings of manual detection of floating objects on the water surface, reduce the workload of workers, and ensure their safety, drones are used to detect floating objects on the water surface. Drone detection of floating objects on the water surface mainly involves collecting a large number of aerial images of the water surface for detection. When processing drone aerial images of floating objects on the water surface, the Faster-RCNN network is often used to process drone aerial images. Based on the accumulation of R-CNN and Fast RCNN networks, the Faster RCNN network has integrated feature extraction, proposal extraction, bounding box regression, and classification into a single network, significantly improving the overall performance, especially in terms of detection speed.

[0005] At present, in terms of the detection category method of floating objects on the water surface in drone aerial photography, the detection and classification are mainly completed through the following steps. First, the VGG-16 network in the Faster-RCNN network is used to extract features of the input image. After extracting the features, the data of the feature map first enters the RPN (Region Proposal Network) network. For RPN, the work done by this network is to select possible candidate areas in the image and distinguish between foreground and background. This information is used to assist the final target detection decision, and then enters the ROI pooling (Region of interest pooling) network. What the ROI Pooling network has to do is to collect the input feature maps (feature maps) and proposals (candidate boxes), and after integrating this information, extract proposal feature maps (candidate feature maps), and send them to the subsequent fully connected layer to determine the target category. Summary of the Invention

[0006] The present invention provides a method for identifying floating objects on the water surface in aerial photography by an unmanned aerial vehicle (UAV) based on a neural network.

[0007] The technical solutions of the present invention are as follows:

[0008] A method for identifying floating objects on the water surface from drone aerial photography based on a neural network comprises the following steps:

[0009] ResNet-50 network generates feature maps: aerial images of floating objects on the water surface are input into the ResNet-50 network to obtain feature maps;

[0010] The CC-FPN network generates a fused feature map: the feature map is input into the CC-FPN network to generate a fused feature map;

[0011] S-RPN network generates candidate boxes: the fused feature map is input into the S-RPN network to generate candidate boxes;

[0012] ROI Align layer generates candidate feature maps: the fused feature map and the candidate box are input into the ROI Align layer to generate candidate feature maps;

[0013] The fully connected layer generates a detection frame: the candidate feature map is input into the fully connected layer to generate a detection frame;

[0014] This detection method uses the more expressive ResNet-50 network, the CC-FPN network with the CAG module, and the more accurate S-RPN network. At the same time, the ROI Align layer with better matching is used as the region of interest layer, making the detection effect of drone aerial images more excellent and the classification of floating objects on the water surface more accurate.

[0015] Preferably, the ResNet-50 network includes a splicing form, which allows each layer of the ResNet-50 network to have a dual-channel splicing, thereby improving the output image efficiency.

[0016] Preferably, the CC-FPN network includes a CAG module, which is used to fuse information from upper and lower layers. The execution process of the CAG module attention mechanism is as follows:

[0017] CA(x)=σ(fc1(AvgPool(x))+fc2(MaxPool(x)))

[0018] Among them, CA(x) is the attention function, σ is the Sigmoid activation function, fc1 and fc2 are feature extraction functions, Avgpool is the average pooling function, which takes the average value of the feature points in the range; maxpool is the maximum pooling function, which takes the maximum value of the feature points in the range; the CC-FPN network adds the CAG module to ensure that the information at low resolution of the image is not diluted too much, so that the output feature map is a feature map that integrates low and high bits.

[0019] Preferably, the S-RPN network generates candidate frames by using a positive and negative sample sampling method to generate anchor frames, which can make the candidate frames more accurate.

[0020] Preferably, the operation of generating the candidate feature map by the ROI Align layer includes the use of a linear interpolation method. The RoIAlign layer solves the drawbacks of multiple quantizations through a bilinear interpolation algorithm, which can significantly improve the detection effect of drone aerial images.

[0021] The present invention also provides a neural network-based device for identifying floating objects on the water surface taken by an unmanned aerial vehicle, comprising:

[0022] ResNet-50 network module, used to extract features of the input image and generate feature maps;

[0023] The CC-FPN network module is used to fuse the input feature maps to generate fused feature maps;

[0024] S-RPN network module, used to generate candidate boxes based on the input fusion map;

[0025] The ROI Align layer module is used to process the input fusion feature map and candidate box to generate a candidate feature map;

[0026] The fully connected layer module is used to process the input candidate feature map and generate a detection box.

[0027] The present invention also provides a device for identifying floating objects on the water surface in aerial photography by unmanned aerial vehicles based on a neural network, comprising a processor and a memory, wherein the processor implements the method for identifying floating objects on the water surface in aerial photography by unmanned aerial vehicles based on a neural network when executing a computer program stored in the memory.

[0028] The present invention also provides a computer-readable storage medium for storing a computer program, wherein when the computer program is executed by a processor, the method for identifying floating objects on the water surface by drone aerial photography based on a neural network is implemented.

[0029] The beneficial effects of the present invention are: the present invention is a method for identifying floating objects on the water surface in aerial photography by unmanned aerial vehicles based on a neural network, and the steps of the detection method include inputting the aerial photography image of floating objects on the water surface into the ResNet-50 network in the Faster-RCNN network to generate a feature map, inputting the feature map into the CC-FPN network to generate a fused feature map, inputting the fused feature map into the S-RPN network to generate a candidate frame, and inputting the fused feature map and the candidate frame into the ROI Align layer to generate a feature candidate map. The detection method uses the ResNet-50 network with stronger expressive ability and the CC-FPN network with high and low bits fused with each other, as well as the S-RPN network with higher accuracy, and at the same time uses the ROI Align layer with better matching as the region of interest layer, so that the detection effect of the unmanned aerial vehicle aerial image is better and the classification of floating objects on the water surface is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] By reading the detailed description of the preferred embodiment below, the solutions and advantages of the present application will become clear to those skilled in the art. The accompanying drawings are only for illustrating the preferred embodiment and are not to be considered as limiting the present invention.

[0031] In the attached figure:

[0032] Figure 1 Flowchart of the method for detecting floating objects on the water surface by drone aerial photography in the embodiment.

[0033] Figure 2 Schematic diagram of the method for detecting floating objects on the water surface by drone aerial photography in an embodiment.

[0034] Figure 3 Schematic diagram of the ResNet-50 network structure in the embodiment.

[0035] Figure 4 Schematic diagram of the CC-FPN network structure in the embodiment.

[0036] Figure 5 Flowchart of the execution of the attention mechanism of the CAG module in the embodiment.

[0037] Figure 6 Schematic diagram of the bilinear interpolation method in the ROI Align layer in the embodiment.

[0038] Figure 7 Graph showing the loss function of the network training in this example.

[0039] Figure 8 2 is a flow chart of the detection device module in the embodiment.

[0040] Figure 9 Flow chart of the detection equipment in the embodiment. DETAILED DESCRIPTION

[0041] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings.

[0042] A method for identifying floating objects on the water surface from drone aerial photography based on neural network, see Figure 1 and Figure 2 , a method for identifying floating objects on the water surface from drone aerial photography based on neural network, the steps are as follows:

[0043] The ResNet-50 network generates a feature map: the aerial image of floating objects on the water surface is input into the ResNet-50 network to obtain a feature map; the CC-FPN network generates a fused feature map: the feature map is input into the CC-FPN network to generate a fused feature map; the S-RPN network generates a candidate box: the fused feature map is input into the S-RPN network to generate a candidate box; the ROI Align layer generates a candidate feature map: the fused feature map and the candidate box are input into the ROI Align layer to generate a candidate feature map; the fully connected layer generates a detection box: the candidate feature map is input into the fully connected layer to generate a detection box.

[0044] Specifically:

[0045] S11, ResNet-50 network generates feature maps

[0046] The aerial image of floating objects on the water surface is input into the ResNet-50 network (ResidualNetwork-50) in the Faster-RCNN network to extract the feature map. Specifically, the original VGG-16 network (Visual Geometry Group-16, convolutional neural network) in the Faster-RCNN network is replaced with the ResNet-50 network, and then the features are adjusted through the activation layer. The input aerial image of floating objects on the water surface is input into the improved ResNet-50 network to obtain the feature map. The ResNet-50 network has a deeper deep learning network layer and stronger expression ability, which is conducive to extracting information-rich feature maps.

[0047] As a preferred embodiment, see Figure 3 , a splicing form will be added to the ResNet-50 network. The splicing form allows a dual-channel splicing to exist between each layer of the ResNet-50 network. Compared with the original network structure, the improved ResNet-50 network takes less time to output images of the same format, increases the efficiency of outputting images, and also improves the network training speed, which significantly improves the detection level of drone aerial images.

[0048] S12, CC-FPN network generates fusion feature map

[0049] The feature map is input into the CC-FPN network in the Faster-RCNN network to generate a fused feature map. Specifically, a CAG module (Channel Attention Guided) is added in parallel to the original FPN network (Feature Pyramid Networks) to form a CC-FPN network. This network is more adaptable to the complexity of drone aerial imagery and facilitates the detection of small targets in aerial imagery. To ensure that the information in low-resolution images is not diluted too much, the original FPN network in the Faster-RCNN network is replaced with a CC-FPN network, and the feature map is input into the CC-FPN network to obtain a fused feature map.

[0050] As a preferred embodiment, see Figure 4 The CAG module is used to fuse the information of the upper and lower layers. It is located in the CC-FPN structure and is spliced ​​with the CC-FPN structure to achieve information fusion of the upper and lower layer feature maps. The CAG module can guide the CC-FPN network to eliminate the overlapping effect, so that the output feature map is a fused feature map of the low-level and high-level layers.

[0051] The execution process of the attention mechanism of the CAG module is as follows:

[0052] CA(x)=σ(fc1(AvgPool(x))+fc2(MaxPool(x)))

[0053] Where CA(x) is the attention function, σ is the Sigmoid activation function, and fc1 and fc2 are fully connected layers. Avgpool is the average pooling function, which takes the average value of feature points within a range; maxpool is the maximum pooling function, which takes the maximum value of feature points within a range.

[0054] See also Figure 5 The specific mechanism flow of the CAG module is as follows: first, the input feature maps are subjected to global average pooling and global maximum pooling operations respectively, and then passed into the fully connected layer respectively. The output weights are obtained through summation processing and Sigmoid activation function, and then the attention feature maps {P2, P3, P4, P5} output by the CAG module are added respectively to obtain the final output result.

[0055] S13, S-RPN network generates candidate boxes

[0056] The fused feature map is input into the S-RPN network in the Faster-RCNN network to generate candidate boxes. Specifically, the original RPN network of the Faster-RCNN network is replaced with the S-RPN network, and the fused feature map is input into the S-RPN network to generate candidate boxes. The accuracy of the S-RPN network is higher than that of the RPN network, which can make the detection results more accurate.

[0057] Specifically, a 512-channel 3×3 convolution is first applied to each fused feature map output by the CC-FPN network to transform the features. Two subnetworks are then added after the feature transformation: one for classification probability and one for bounding box regression. Finally, a coordinate system with classification scores and bounding box regression is obtained.

[0058] As a preferred implementation scheme, the S-RPN network uses the positive and negative sample sampling method to generate anchor boxes and sample positive and negative samples. When training the target detector, the positive and negative samples are sampled according to the number of centers. If the number of centers is ≥ k, it is considered a positive sample and marked as 1, otherwise it is a negative sample and marked as 0. Negative samples are ignored when training bounding box regression. After multiple experiments, the value of k is set to 0.5, which not only improves the detection speed but also ensures the detection rate of small targets.

[0059] Furthermore, the distance from the four sides of the anchor box to the center of the true value box is l, r, t, b, and given the side lengths of the anchor box (c, d, e, f) and the side lengths of the true value box (w, h, x, y), the anchor box and the true value box are rectangular, l, r, t, b are the distances from the upper boundary, lower boundary, left boundary, and right boundary of the anchor box to the center of the true value box, c, d, e, f are the upper side length, lower side length, left side length, and right side length of the anchor box, w, h, x, y are the upper side length, lower side length, left side length, and right side length of the true value box, respectively. The formula is as follows:

[0060] l=x-x1,t=y-y1,r=x2-x,b=y2-y

[0061] The formula for obtaining the output image based on center sampling is as follows:

[0062]

[0063] After the positive and negative samples are sampled to generate the anchor box, the softmax layer inside the S-RPN outputs the probability of whether the anchor box contains the target and uses this to decide whether to retain the candidate box.

[0064] S14, ROI Align layer generates candidate feature maps

[0065] The fused feature map and the candidate box are input into the ROI Align layer in the Faster-RCNN network to obtain the candidate feature map. Specifically, the fused feature map and the candidate box are input into the region of interest layer, and the ROI Align layer is used to replace the original ROI Pooling layer in the Faster-RCNN network. As a result, the candidate feature map is output; the ROI Align layer ((Region of interest Align, a special layer for target detection)) can well solve the problem of regional mismatch caused by the double quantization in the ROI Pooling operation. Replacing the ROI Pooling layer with the ROI Align layer can improve the accuracy of the detection model.

[0066] As a preferred implementation method, bilinear interpolation is used in the ROI Align layer to obtain a feature map of a certain size. The bilinear interpolation method is to divide the region of interest into m×m grids, each grid has 4 sampling points, perform bilinear interpolation calculations on each sampling point, and finally perform a maximum pooling operation on the sampling points to obtain a candidate feature map. The bilinear interpolation method can significantly improve the detection speed.

[0067] See also Figure 6 The bilinear interpolation method is as follows: on the x and y coordinate axes, assuming that point P is the interpolation point to be calculated, A 11 ,A 12 ,A21 ,A 22 The coordinates of the four points are known, that is, x1, x2, y1, and y2 are known. First, linear interpolation is performed in the x direction. From point A 11 ,A 12 Do linear interpolation to get B1, and then you know the value of x0. Then from point A 11 ,A 12 Perform linear interpolation to obtain B2. The linear interpolation algorithm can be used to obtain the interpolation values ​​corresponding to B1 and B2. The formula is as follows:

[0068]

[0069]

[0070] After obtaining f(B1) and f(B2), perform linear interpolation in the y direction to obtain the interpolation value of point P. The formula is as follows:

[0071]

[0072] Finally, substituting the values ​​solved above into the bilinear interpolation formula is:

[0073]

[0074] The RoI Align layer solves the drawback of the RoI Pooling layer requiring two quantizations through a bilinear interpolation algorithm, significantly improving the detection effect of drone aerial images.

[0075] S15, fully connected layer generates detection box

[0076] First, the candidate feature map is sent to the fully connected layer in the Faster-RCNN network for further fusion and extraction of features, and then classification and bounding box regression are performed. The category of each candidate box is calculated through the fully connected layer and the softmax function, and the category probability is output. The offset of the candidate box is obtained by bounding box regression to obtain a more accurate detection frame.

[0077] Then, the error between the detection box and the actual annotation box is calculated to obtain the detection rate.

[0078] In this embodiment, the inventors collected 1400 images of floating objects on the water surface, and after data enhancement, the average precision AP (Average Precision) values ​​of various types of floating objects on the water surface were obtained by using the detection method provided by the present invention (see Table 1), and the loss function of network training (see Figure 7 ).

[0079] See Table 1 for the AP values ​​of the proposed method, the SSD algorithm, and the Faster-RCNN algorithm for various types of floating objects on the water. These results demonstrate that the proposed algorithm has good accuracy and robustness for detecting floating objects on the water surface from drone aerial photography, facilitating subsequent salvage and classification of floating objects on the water surface.

[0080] Table 1

[0081]

[0082] See also Figure 7 AP (Average Precision) is the average precision, that is, the area under the PR curve is the average of the Precisioon (precision) values ​​on the PR curve; IoU (Intersection over Union) is a simple measurement standard. As long as a task that obtains a prediction in the output can be measured by IoU, IoU is used to measure the degree of overlap between the predicted box and the true box in target detection, that is, the degree of overlap between the predicted box obtained by S15 and the true box marked on the image data. The formula is as follows:

[0083]

[0084]

[0085]

[0086]

[0087] Where P stands for precision and R stands for recall. TP represents the number of positive samples identified as positive samples, FN represents the number of positive samples identified as negative samples, and FP represents the number of negative samples identified as positive samples. In the IoU calculation, A and B are the predicted box and the ground truth box, respectively.

[0088] Specifically, as the number of horizontal axis iterations continues to increase, the error value gradually tends to a stable value, proving that the accuracy of the entire network gradually reaches a stable value.

[0089] In this embodiment, the loss function of the network training of the detection method provided by the present invention is calculated, see Figure 7 , (1) is the S-RPN bounding box regression loss, (2) is the S-RPN classification loss, (3) is the bounding box regression loss, (4) is the classification loss, and (5) is the total loss. The classification loss function used in this embodiment is:

[0090]

[0091] Where H is the classification loss function, is the true label, oi is the predicted value, and the default logarithm is the base e. The regression loss function used is:

[0092]

[0093] Where x represents the difference between the actual value and the predicted value.

[0094] Based on the neural network-based method for identifying floating objects on the water surface in aerial photography by a UAV, see Figure 8 The embodiment of the present invention further provides a device for identifying floating objects on the water surface in aerial photography by a drone based on a neural network, comprising:

[0095] ResNet-50 network module, used to extract features of the input image and generate feature maps;

[0096] The CC-FPN network module is used to fuse the input feature maps to generate fused feature maps;

[0097] S-RPN network module, used to generate candidate boxes based on the input fusion map;

[0098] The ROI Align layer module is used to process the input fusion feature map and candidate box to generate a candidate feature map;

[0099] The fully connected layer module is used to process the input candidate feature map and generate a detection box.

[0100] As described above, the present invention discloses a device for identifying floating objects on the water surface in drone aerial photography based on a neural network. By establishing a ResNet-50 network module, a CC-FPN network module, an S-RPN network module, an ROI Align layer module, and a fully connected layer module, the present invention implements all steps of a method for identifying floating objects on the water surface in drone aerial photography based on a neural network. The device can be separately set in a hardware device.

[0101] In addition, the embodiment of the present invention also provides a device for identifying floating objects on the water surface by drone aerial photography based on a neural network, see Figure 9 , including: a processor and a memory, wherein when the processor executes the computer program stored in the memory, it implements a method for identifying floating objects on the water surface in drone aerial photography based on a neural network disclosed in the above embodiment.

[0102] At the same time, the present invention also provides a computer-readable storage medium for storing a computer program, wherein when the computer program is executed by a processor, it implements a method for identifying floating objects on the water surface in drone aerial photography based on a neural network disclosed in the above embodiment.

[0103] The above is a detailed introduction to the technical solutions provided by this application. Specific examples are used herein to illustrate the principles and implementation methods of this application. The description of the above embodiments is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the contents of this specification should not be understood as limiting this application.

[0104] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present invention, and will not be described in detail here.

[0105] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying floating objects on the water surface from drone aerial photography based on a neural network, characterized in that: The following steps are involved: ResNet-50 network generates feature maps: Aerial images of floating objects on the water surface are input into the ResNet-50 network to generate feature maps. The ResNet-50 network has a splicing form, which enables a dual-channel splicing between each layer of the ResNet-50 network. The CC-FPN network generates a fused feature map: The feature map is input into the CC-FPN network to generate a fused feature map. The CC-FPN network is formed by adding a CAG module in parallel to the original FPN network. S-RPN network generates candidate boxes: Input the fused feature map into the S-RPN network to generate candidate boxes; ROI Align layer generates candidate feature maps: the fused feature map and the candidate box are input into the ROI Align layer to generate candidate feature maps; The fully connected layer generates a detection box: the candidate feature map is input into the fully connected layer to generate a detection box.

2. The method for identifying floating objects on the water surface by drone aerial photography based on a neural network according to claim 1, characterized in that: The CAG module is used to fuse information from the upper and lower layers. The execution process of the CAG module attention mechanism is as follows: That is the attention function, is the Sigmoid activation function, is the feature extraction function, Avgpool It is the average pooling function, which takes the average value of the feature points within the range; maxpool It is the maximum pooling function, which takes the maximum value of the feature points within the range.

3. The method for identifying floating objects on the water surface by drone aerial photography based on a neural network according to claim 1, characterized in that: The operation of the S-RPN network to generate candidate frames includes generating anchor frames using positive and negative sample sampling methods.

4. The method for identifying floating objects on the water surface by drone aerial photography based on a neural network according to claim 1, characterized in that: The operation of generating candidate feature maps in the ROI Align layer includes the use of linear interpolation method.

5. A device for identifying floating objects on the water surface from drone aerial photography based on a neural network, characterized in that: include: ResNet-50 network module, used to extract features of the input image and generate feature maps; There is a splicing form in the ResNet-50 network, which enables a dual-channel splicing between each layer of the ResNet-50 network; The CC-FPN network module is used to fuse the input feature maps to generate a fused feature map. The CC-FPN network is formed by adding a CAG module in parallel to the original FPN network. S-RPN network module, used to generate candidate boxes based on the input fusion feature map; The ROI Align layer module is used to process the input fusion feature map and candidate box to generate a candidate feature map; The fully connected layer module is used to process the input candidate feature map and generate a detection box.

6. A neural network-based UAV aerial photography floating object identification device, characterized in that: The method comprises a processor and a memory, wherein when the processor executes the computer program stored in the memory, it implements the neural network-based method for identifying floating objects on the water surface in drone aerial photography as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that Used to store a computer program, wherein when the computer program is executed by a processor, it implements the neural network-based method for identifying floating objects on the water surface in drone aerial photography according to any one of claims 1 to 4.

Citation Information

Patent Citations

  • Pedestrian re-identification method based on multi-scale convolution feature fusion

    CN111709311A

  • Method and device for identifying target object in cloud federation image

    CN112733686A