A deep learning-based unmanned aerial vehicle rockfall target detection method and device

Abstract

In order to prevent the safety hidden trouble of mountain rockfall from occurring in the construction site and reduce unnecessary personnel casualties, the unmanned aerial vehicle rockfall target detection method and device based on deep learning, the method first collects the construction area image through the unmanned aerial vehicle, secondly uses the target detection technology to mark the stones in the image with anchor frames, then calculates the anchor frame pixel center points of all the stones, obtains the Euclidean distance of the anchor frames of any two stones in the image according to the Euclidean distance method, establishes the Euclidean distance information set of the reference image, records the three-dimensional coordinate system information of the unmanned aerial vehicle at the same time, and simultaneously regards the image processed at the same time as the reference image. Finally, when the coordinates of the unmanned aerial vehicle are the same, the Euclidean distance of any two points of the anchor frame center points of the stones in the construction area image collected subsequently is not in the Euclidean distance set of the reference image, and the unmanned aerial vehicle sends a warning to the background management system.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and device for detecting falling rocks by an unmanned aerial vehicle (UAV) based on deep learning. Background Art

[0002] Rockfall is a common safety hazard on construction sites. If not detected and addressed promptly, it can cause serious harm and damage to workers and equipment. Traditional rockfall monitoring methods rely primarily on manual inspections or the installation of surveillance cameras. However, these methods suffer from limited monitoring range, low efficiency, and high costs.

[0003] Traditional construction area target detection requires the installation of visible light image acquisition equipment in the construction area, and the acquisition range of the visible light image acquisition equipment must cover the entire construction site area. The use of drone image detection can not only quickly realize the inspection of the construction site area, but also detect loopholes and blind spots in the control of the image acquisition equipment in the construction site area. However, in addition to the variable shooting angles and heights, drone aerial images also have the problems of large shooting field of view, small target proportion, complex background, susceptibility to lighting factors and background complexity.

[0004] Regarding the problems with drones, patent document CN115859056A discloses a drone target detection method based on a neural network, which uses a time domain matrix to improve the accuracy of drone target detection; patent document CN112668644A discloses a drone aerial photography target detection method based on an improved SSD algorithm, which implements drone aerial photography detection based on the SSD algorithm; both of the above solutions are conducive to achieving the purpose of drone detection of fallen rocks, but both have the problems of slow detection speed and insufficient detection accuracy.

[0005] As for rockfall detection on construction sites, patent document CN115035176A discloses a method and system for detecting multi-target landslides and rockfalls using multi-camera vision, and patent document CN112949484A discloses a real-time high-speed railway rockfall detection method based on aggregated channel features and texture features, using aggregated feature channels to detect rockfall targets. Both of these methods require a large number of visible light image acquisition devices to detect rockfalls, and are unable to detect blind spots in a timely manner. Summary of the Invention

[0006] The purpose of the present invention is to overcome the above-mentioned deficiencies in the prior art and to provide a

[0007] To achieve the above object, the technical solution of the present invention is:

[0008] In a first aspect, the present invention provides a method for detecting falling rocks from a drone based on deep learning, the method comprising:

[0009] Obtain monitoring images of the construction area collected by the drone;

[0010] The collected monitoring image is used as the input of the target detection neural network model to mark all the stones in the monitoring image with anchor boxes to obtain the target detection result map.

[0011] Mark the result image as the reference image and record the three-dimensional coordinate information of the drone at this time;

[0012] Calculate the pixel center points of all anchor boxes of the reference image;

[0013] Calculate the Euclidean distance between the pixel centers of all anchor boxes to form the Euclidean distance information set of the reference image;

[0014] Keep the three-dimensional coordinate information of the drone unchanged, continue to obtain the monitoring image as the detection map, and obtain the prediction result map through the target detection neural network model;

[0015] Calculate the Euclidean distance between the pixel centers of all anchor boxes in the prediction result image to obtain the Euclidean distance information set of the prediction image;

[0016] The Euclidean distance set of the predicted image is matched with the Euclidean distance information set of the reference image to determine whether there is any mismatched distance information. When the Euclidean distance information of the predicted image does not match the Euclidean distance information of the reference image, an alarm message is issued.

[0017] Furthermore, the target detection neural network model includes DarkNet, a hybrid feature pyramid module, a loss function, and a YOLO detection head;

[0018] After the input surveillance image enters the target detection neural network model, the stone features of the image are first extracted through DarkNet. Secondly, the hybrid feature pyramid module is used to improve the detection accuracy of the model and the contextual semantic information of the image to generate three downsampled feature maps of different sizes. Finally, the three downsampled feature maps of different sizes are passed through the loss function, and the YOLO detection head is used to obtain the target detection result map with anchor box.

[0019] Furthermore, the hybrid feature pyramid module includes a spatial pyramid and a hollow pyramid;

[0020] The hybrid feature pyramid module is used to improve the detection accuracy of the model and the contextual semantic information of the image to generate feature maps of three different sizes, including:

[0021] After the monitoring image passes through the DarkNet backbone network, it generates a 52-fold downsampled feature map and enters the hybrid feature pyramid module. First, it needs to pass through a 1x1 convolution module to extract the feature map information, then obtain the effective feature channel information through the spatial pyramid, and finally pass through a 3x3 convolution to output the resulting feature image. The resulting feature image is processed by a 1x1 convolution and then upsampled to restore its size to the size of the 16-fold downsampled feature map. It is then fused with the residual module that outputs 16-fold downsampling in DarkNet. The feature map after the feature extraction is processed by 1x1 convolution, and enters the spatial pyramid module to output the feature map. The feature map output by the spatial pyramid uses 3x3 convolution to extract image information. At this time, two branches are generated. One branch directly outputs the processed 16-fold down-sampled feature map, and the other branch is up-sampled again after 1x1 convolution and combined with the output of the residual module that outputs the 8-fold down-sampled feature map in the backbone network. After the hollow pyramid, the processed 8-fold down-sampled feature map is output. The above operation is then used to finally obtain the processed 16-fold, 8-fold, and 4-fold down-sampled feature maps.

[0022] Furthermore, the network depth of DarkNet is 59 layers.

[0023] Furthermore, the loss function predicts the processed downsampled feature map to determine the area where the stone is located in the image.

[0024] Furthermore, the loss function is LOSS GIOU , as shown in formula (1):

[0025] LOSS GIOU =LOSS LOc +LOSS conf +LOSS class (1)

[0026] In formula 1, LOSS LOc Refers to position loss, LOSS conf Refers to confidence loss, LOSS class Refers to the category loss function, and the GIOU function formula is used to calculate the accuracy of the real box and the marked anchor box as shown in formula (2):

[0027]

[0028] In formula 2, the pixel coordinates of the predicted box are Marked as B p , are the pixel coordinates of the upper left and lower right corners of the predicted box, and the pixel coordinates of the real box are (x1, y1, x2, y2), where (x1, y1, x2, y2) are the pixel coordinates of the upper left and lower right corners of the predicted box, respectively.

[0029] Furthermore, calculating the pixel center points of all anchor frames of the reference image includes:

[0030] By calculating the predicted box pixels The distance from the center point is obtained in Get the pixel center coordinates of all predicted anchor boxes in the benchmark image.

[0031] Furthermore, the calculation of the Euclidean distances between pixel centers of all anchor frames to form the reference graph Euclidean distance information set includes:

[0032] The Euclidean distance is calculated by randomly selecting the pixel center coordinates of two anchor boxes, and the Euclidean distance set of the reference image is obtained by using the Euclidean distance of all the randomly selected pixel center coordinates. The Euclidean distance calculation formula is shown in Equation 3:

[0033]

[0034] in and are the pixel center coordinates of two randomly selected anchor boxes. After calculation, the Euclidean distance information of any two anchor boxes in the image will be obtained.

[0035] In the second aspect, the present invention provides a deep learning-based drone rockfall target detection device, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor, characterized in that the processor implements the steps of the above-mentioned method when executing the computer program.

[0036] In a third aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described above are implemented.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The present invention utilizes drones for patrols, which can achieve all-round monitoring of the construction site with a wide monitoring range and high monitoring efficiency, while also reducing the workload and cost of manual inspections; by using advanced image processing algorithms, the construction site image information can be processed efficiently and accurately, and falling rock targets can be identified, thereby improving the accuracy and timeliness of monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 Flowchart of the deep learning-based UAV rockfall target detection method provided in Example 1 of the present invention;

[0040] Figure 2 Schematic diagram of the composition of the target detection neural network model;

[0041] Figure 3 Flowchart of network design for deep learning-based UAV construction site rockfall target detection neural network;

[0042] Figure 4 This is the flow chart of the hybrid feature pyramid module;

[0043] Figure 5 Schematic diagram of the composition of the deep learning-based drone rockfall target detection device provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0044] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0045] Example 1:

[0046] In order to prevent the safety hazards of mountain rockfall at construction sites and reduce unnecessary casualties, this paper proposes a UAV rockfall target detection method based on deep learning. Figure 1 As shown, the method specifically includes the following steps:

[0047] 101. Obtain monitoring images of the construction area collected by the drone;

[0048] In this step, the image acquisition device (such as Sony A7RIII) carried by the drone (such as DJI Mavic 2 Pro) collects images of the hillside stones near the construction site to ensure that the drone can capture the entire construction area. The second step is to start the video shooting mode and intercept images at a fixed frequency. After the images are collected, they are transmitted to the lightweight computing device (such as NVIDIA Jetson Nano) carried by the drone.

[0049] The collected surveillance image is used as the input of the target detection neural network model to mark all the stones in the surveillance image with anchor boxes to obtain the target detection result map;

[0050] A target detection neural network model is integrated on the lightweight computing device. The lightweight computing device first performs denoising on the transmitted image to separate the stone targets in the image from the background, and then uses the target detection neural network model to mark the positions of all the stones in the image with anchor frames, thereby obtaining a target detection result image with anchor frames.

[0051] 103. Mark the result image as a reference image and record the three-dimensional coordinate information of the UAV at this time;

[0052] 104. Calculate the pixel center points of all anchor boxes of the reference image;

[0053] 105. Calculate the Euclidean distance of the pixel center points between all anchor boxes to form the Euclidean distance information set of the reference image;

[0054] 106. Keep the three-dimensional coordinate information of the UAV unchanged, continue to obtain the monitoring image as the detection map, and obtain the prediction result map through the target detection neural network model;

[0055] 107. Calculate the Euclidean distance between the pixel centers of all anchor boxes in the prediction result image, and obtain the Euclidean distance information set of the prediction image;

[0056] 108. The predicted image's Euclidean distance set is matched with the reference image's Euclidean distance information set to determine if any mismatches occur. If the predicted image's Euclidean distance information does not match the reference image's Euclidean distance information, an alarm is issued, indicating a potential rockfall risk in the construction site. If the drone's 3D coordinate information changes, the above steps must be repeated for repositioning.

[0057] It can be seen that the present invention uses drones for inspections to achieve all-round monitoring of the construction site, with a wide monitoring range and high monitoring efficiency, and can also reduce the workload and cost of manual inspections; by using advanced image processing algorithms, the construction site image information can be processed efficiently and accurately, and falling rock targets can be identified, thereby improving the accuracy and timeliness of monitoring.

[0058] Specifically, if Figure 2 As shown, the above-mentioned target detection neural network model consists of four parts: backbone network, network neck, loss function, and detection head. The model is improved based on YOLOv8. The detailed algorithm flow chart is as follows Figure 3 As shown, it consists of DarkNet and hybrid feature pyramid modules and loss functions as well as YOLO detection head.

[0059] The DarkNet backbone network consists of four residual convolutional feature extraction modules, forming a four-scale detector. Through convolutional stacking, DarkNet's network depth reaches 59 layers, enabling the extraction of downsampled feature maps at 4x, 8x, 16x, and 52x scales. This facilitates the recognition of small targets by high-altitude drones and improves network recognition accuracy. Therefore, this example uses DarkNet59 instead of the YOLOv8 backbone network to extract feature information about stones in the construction area.

[0060] The hybrid feature pyramid module is an innovation of the target detection neural network model, also known as the neural network neck module. This method first proposed that the target detection neural network neck module can be implemented by the hybrid feature pyramid module. The implementation principle diagram of this module is shown in the figure below. Figure 4 As shown in the figure, the spatial pyramid is used to receive the final output of DarkNet59, and the outputs of the other three residual network modules are received by the hollow pyramid module. After passing through DarkNet59, the image generates a 52-fold downsampled feature map that enters the neck module. First, it needs to pass through a 1x1 convolution module to extract the feature map information. Secondly, the spatial pyramid is used to obtain the effective feature channel information. Finally, a 3x3 convolution is performed to output the resulting feature image. After the resulting feature image is processed by a 1x1 convolution, it is upsampled and restored to the size of the 16-fold downsampled feature map. The feature fusion is then performed with the residual module that outputs the 16-fold downsampled feature map in DarkNet. The fused feature map is processed by a 1x1 convolution for feature extraction and enters the spatial pyramid module to output the feature map. The spatial pyramid module can increase the receptive field to obtain image detail information and enhance the model's ability to extract contextual image feature information. The feature map output by the spatial pyramid is then used to extract image information using 3x3 convolution. At this time, two branches are generated. One branch directly outputs the processed 16x downsampled feature map, and the other branch is upsampled again after 1x1 convolution and combined with the output of the residual module that outputs the 8x downsampled feature map in the backbone network. After the hollow pyramid, the processed 8x downsampled feature map is output. The above operation is then used to finally obtain the processed 16x, 8x, and 4x downsampled feature maps. The three maps of different feature sizes are input into the loss function, and finally the YOLO detection head is used to realize the drone's stone target detection in the construction area of ​​the construction site.

[0061] Loss function, this method uses the loss function to predict the processed downsampled feature map and determine the area where the stone is located in the image. The loss function used in this method is LOSS GIOU , this function can be used to calculate the distance between the predicted bounding box and the true bounding box, thereby reducing the error. The function is shown in formula (1):

[0062] LOSS GIOU =LOSS LOc +LOSS conf +LOSS class (1)

[0063] In formula 1, LOSS LOc Refers to position loss, LOSS conf Refers to confidence loss, LOSS classRefers to the category loss function, using the GIOU function formula to calculate the accuracy of the real box and the marked anchor box as shown in formula (2)

[0064]

[0065] The pixel coordinates of the prediction box in formula 2 are Marked as B p , are the pixel coordinates of the upper left and lower right corners of the predicted box, and the pixel coordinates of the real box are (x1, y1, x2, y2), where (x1, y1, x2, y2) are the pixel coordinates of the upper left and lower right corners of the predicted box, respectively.

[0066] The prediction head part uses the prediction head module of the YOLOv8 network model. After generating three downsampled feature maps of different sizes, image fusion is performed to realize the rockfall target detection neural network of the construction area photographed by the drone. The image with the predicted anchor frame is output and set as the reference image. The three-dimensional coordinate position of the drone at the time of image acquisition is recorded.

[0067] The image is processed by the neural network model to obtain the reference image. Since there are multiple anchor boxes in the image, the center point of the anchor box set by this method is calculated by calculating the pixel points of the prediction box. The distance from the center point is obtained in Obtain the pixel center coordinates of all predicted anchor boxes in the reference image. Then, calculate the Euclidean distance by randomly selecting the pixel center coordinates of two anchor boxes. Then, calculate the Euclidean distance set of all randomly selected pixel center coordinates using the Euclidean distance. The Euclidean distance calculation formula is shown in Equation 3:

[0068]

[0069] in and The pixel center coordinates of two randomly selected anchor frames are calculated, and the Euclidean distance information of any two anchor frames in the image is obtained after calculation. When the three-dimensional coordinate information of the drone remains unchanged, the detection image will obtain a new prediction result map with anchor frames through the neural network model designed by this patent, and the Euclidean distance of any two anchor frames in the prediction result map is calculated. When the Euclidean distance result of the prediction result map cannot be matched in the Euclidean distance set of the reference map, the drone will issue an alarm signal and transmit the image to the background detection system at the same time. The drone will issue a warning to the background management system, indicating that there may be a risk of falling rocks in the construction site area.

[0070] In summary, the present invention has the following technical advantages compared with the prior art:

[0071] To effectively improve the recognition performance of drones for intelligent rockfall detection at construction sites, this method proposes a YOLOv8-based drone rockfall detection method based on deep learning target detection neural network technology. This method uses the target detection network to obtain a baseline output result map with anchor frames, records the drone's 3D coordinate information at that time, calculates the Euclidean distance between any two anchor frames in the baseline map, and outputs the Euclidean distance information set of the baseline map. Images captured using the same drone 3D coordinate information are then used to generate a predicted result map with anchor frames. The Euclidean distance between any two anchor frames in the predicted result map is then calculated. If the Euclidean distance result does not match the Euclidean distance information set of the baseline map, an alarm is issued and the captured image is transmitted to the backend signal.

[0072] In the deep learning neural network model designed by this method, DarkNet59 is first used as the backbone network. This method effectively extracts image information and can output downsampled feature maps of different sizes. Secondly, a hybrid feature pyramid module is used at the neck of the network to combine a dilated convolutional pyramid with a spatial pyramid. This helps extract both pixel features and contextual semantic information from the image, thereby improving the accuracy of the object detection model. The GIOU loss function is then used to facilitate rapid model convergence and improve model accuracy. Finally, the YOLOv8 detection head is used to output the prediction result image with anchor boxes.

[0073] Another innovation of this method is to calculate the Euclidean distance between any two anchor frames based on the pixel centers of all anchor frames in the reference image, thereby obtaining the Euclidean distance information set of the reference image. While the drone's 3D coordinate position remains unchanged, the Euclidean distance between any two anchor frames is calculated based on the pixel centers of all anchor frames in the detection image. An alarm is issued if the Euclidean distance result of the detection image does not match the Euclidean distance information set of the reference image.

[0074] Example 2:

[0075] See Figure 5 As shown, the deep learning-based UAV rockfall target detection device provided in this embodiment includes a processor 51, a memory 52, and a computer program 53 stored in the memory 52 and executable on the processor 51, such as a deep learning-based UAV rockfall target detection program. When the processor 51 executes the computer program 53, the steps of the above-mentioned embodiment 1 are implemented, such as Figure 1 Steps shown.

[0076] Exemplarily, the computer program 53 may be divided into one or more modules / units, which are stored in the memory 52 and executed by the processor 51 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program 53 in the deep learning-based UAV rockfall target detection device.

[0077] The deep learning-based UAV rockfall target detection device can be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The deep learning-based UAV rockfall target detection device can include, but is not limited to, a processor 51 and a memory 52. ​​Those skilled in the art will understand that Figure 5 It is only an example of a deep learning-based UAV rockfall target detection device and does not constitute a limitation of the deep learning-based UAV rockfall target detection device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the deep learning-based UAV rockfall target detection device may also include input and output devices, network access devices, buses, etc.

[0078] The processor 51 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0079] The memory 52 can be an internal storage unit of the deep learning-based drone rockfall target detection device, such as a hard disk or memory of the deep learning-based drone rockfall target detection device. The memory 52 can also be an external storage device of the deep learning-based drone rockfall target detection device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the deep learning-based drone rockfall target detection device. Furthermore, the memory 52 can also include both the internal storage unit of the deep learning-based drone rockfall target detection device and an external storage device. The memory 52 is used to store the computer program and other programs and data required by the deep learning-based drone rockfall target detection device. The memory 52 can also be used to temporarily store data that has been output or is to be output.

[0080] Example 3:

[0081] This embodiment provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the method described in Example 1 are implemented.

[0082] The computer-readable medium shown can be any device that can contain, store, communicate, propagate or transmit the program for use with an instruction execution system, device or apparatus or in conjunction with such instruction execution system, device or apparatus. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion having one or more wires (electronic device), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium can even be paper or other suitable medium on which the program can be printed, for example, by optically scanning the paper or other medium, then editing, interpreting or, if necessary, processing in other suitable ways to obtain the program electronically and then storing it in a computer memory.

[0083] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made based on the essence of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A deep learning-based UAV rockfall target detection method, characterized in that: The method comprises: Obtain monitoring images of the construction area collected by the drone; The collected surveillance image is used as the input of the target detection neural network model to mark all the stones in the surveillance image with anchor boxes to obtain the target detection result map; Mark the result image as the reference image and record the three-dimensional coordinate information of the drone at this time; Calculate the pixel center points of all anchor boxes of the reference image; Calculate the Euclidean distance between the pixel centers of all anchor boxes to form the Euclidean distance information set of the reference image; Keep the three-dimensional coordinate information of the drone unchanged, continue to obtain the monitoring image as the detection map, and obtain the prediction result map through the target detection neural network model; Calculate the Euclidean distance between the pixel centers of all anchor boxes in the prediction result image to obtain the Euclidean distance information set of the prediction image; The Euclidean distance set of the predicted image is matched with the Euclidean distance information set of the reference image to determine whether there is any mismatched distance information. When the Euclidean distance information of the predicted image does not match the Euclidean distance information of the reference image, an alarm message is issued.

2. The deep learning-based UAV rockfall target detection method according to claim 1, characterized in that: The target detection neural network model includes DarkNet, a hybrid feature pyramid module, a loss function, and a YOLO detection head; After the input surveillance image enters the target detection neural network model, the stone features of the image are first extracted through DarkNet. Secondly, the hybrid feature pyramid module is used to improve the detection accuracy of the model and the contextual semantic information of the image to generate three downsampled feature maps of different sizes. Finally, the three downsampled feature maps of different sizes are passed through the loss function, and the YOLO detection head is used to obtain the target detection result map with anchor box.

3. The deep learning-based UAV rockfall target detection method according to claim 2, wherein: The hybrid feature pyramid module includes a spatial pyramid and a hollow pyramid; The hybrid feature pyramid module is used to improve the detection accuracy of the model and the contextual semantic information of the image to generate feature maps of three different sizes, including: After the monitoring image passes through the DarkNet backbone network, it generates a 52-fold downsampled feature map and enters the hybrid feature pyramid module. First, it needs to pass through a 1x1 convolution module to extract the feature map information, then obtain the effective feature channel information through the spatial pyramid, and finally pass through a 3x3 convolution to output the resulting feature image. The resulting feature image is processed by a 1x1 convolution and then upsampled to restore its size to the size of the 16-fold downsampled feature map. It is then fused with the residual module that outputs 16-fold downsampling in DarkNet. The feature map after the feature extraction is processed by 1x1 convolution, and enters the spatial pyramid module to output the feature map. The feature map output by the spatial pyramid uses 3x3 convolution to extract image information. At this time, two branches are generated. One branch directly outputs the processed 16-fold down-sampled feature map, and the other branch is up-sampled again after 1x1 convolution and combined with the output of the residual module that outputs the 8-fold down-sampled feature map in the backbone network. After the hollow pyramid, the processed 8-fold down-sampled feature map is output. The above operation is then used to finally obtain the processed 16-fold, 8-fold, and 4-fold down-sampled feature maps.

4. The deep learning-based UAV rockfall target detection method according to claim 2 or 3, wherein: The network depth of DarkNet is 59 layers.

5. The deep learning-based UAV rockfall target detection method according to claim 2 or 3, wherein: The loss function predicts the processed downsampled feature map to determine the area where the stone is located in the image.

6. The deep learning-based UAV rockfall target detection method according to claim 5, characterized in that: The loss function is LOSS GIOU , as shown in formula (1): LOSS GIOU =LOSS LOc +LOSS conf +LOSS class (1) In formula 1, LOSS LOc Refers to position loss, LOSS conf Refers to confidence loss, LOSS class Refers to the category loss function, and the GIOU function formula is used to calculate the accuracy of the real box and the marked anchor box as shown in formula (2): In formula 2, the pixel coordinates of the predicted box are Marked as B p , are the pixel coordinates of the upper left and lower right corners of the predicted box, and the pixel coordinates of the real box are (x1, y1, x2, y2), where (x1, y1, x2, y2) are the pixel coordinates of the upper left and lower right corners of the predicted box, respectively.

7. The deep learning-based UAV rockfall target detection method according to claim 1, wherein: The pixel center points of all anchor boxes of the calculation reference image include: By calculating the predicted box pixels The distance from the center point is obtained in Get the pixel center coordinates of all predicted anchor boxes in the benchmark image.

8. The deep learning-based UAV rockfall target detection method according to claim 7, wherein: The calculation of the Euclidean distances between pixel centers of all anchor frames to form the reference graph Euclidean distance information set includes: The Euclidean distance is calculated by randomly selecting the pixel center coordinates of two anchor boxes, and the Euclidean distance set of the reference image is obtained by using the Euclidean distance of all the randomly selected pixel center coordinates. The Euclidean distance calculation formula is shown in Equation 3: in and are the pixel center coordinates of two randomly selected anchor boxes. After calculation, the Euclidean distance information of any two anchor boxes in the image will be obtained.

9. A deep learning-based UAV rockfall target detection device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 3 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 are implemented.

Images