Construction method of visual detection model for unmanned aerial vehicle, detection method and fishing detection system

Abstract

The application relates to the technical field of visual detection, and provides a visual detection model for a drone, a detection method and a fishing detection system.The visual detection model for the drone provided by the application adopts a YOLOV11 structure, improves a C2PSA module into a C2MSPK module, aggregates local information by using deep separable convolution, then captures multi-scale context by using multi-branch deep separable convolution, and models the relationship between different channels by using 1*1 convolution, so that the model performance is improved, and the detection capability for small targets is also enhanced.Meanwhile, a reparameterization context anchor frame attention component is adopted, a serial structure is changed into a parallel structure, so that the effect can be guaranteed, and the calculation complexity of the model can be reduced.The visual detection model for the drone provided by the application improves the original model by introducing multi-scale attention and structure reparameterization ideas, and improves the target detection task with a large scale span.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology, and in particular to a method for constructing a visual inspection model for unmanned aerial vehicles (UAVs), an inspection method, and a fishing inspection system. Background Technology

[0002] Fishing, as a recreational activity, has become increasingly popular in recent years, even developing into an industry. However, with the development of fishing activities, illegal fishing has become a widespread phenomenon, causing serious damage to the ecosystems and natural resources of rivers and lakes. Furthermore, the indiscriminate dumping of waste during illegal fishing significantly impacts the sustainable use of aquatic resources. Therefore, it is necessary to strengthen the management of fishing activities, guide people to fish legally in designated areas, and monitor and address fishing activities outside designated areas, thereby protecting water resources and maintaining ecological balance.

[0003] Current measures for monitoring and handling illegal fishing mainly rely on staff patrols and the installation of fixed monitoring equipment in key areas. These methods have limited effectiveness. With the development of drones, using them to patrol relevant waters and identify fishing behavior has become an important tool. However, current drone-based fishing detection suffers from low resolution of people in the drone's view, resulting in low identification accuracy. Furthermore, fishing rods are often long and thin, creating a significant size difference between them and human figures, making simultaneous identification and detection difficult. Summary of the Invention

[0004] In view of the problems of the large size difference between fishing rods and human figures in existing drone detection of fishing behavior, making it difficult to identify and detect them simultaneously, this application provides a visual recognition model for drones.

[0005] The method for constructing a visual inspection model for UAVs provided in this application embodiment is based on the YOLOV11 (YouOnly Look Once version 11) structure, and replaces the C2PSA module of the YOLOV11 structure with the C2MSPK module;

[0006] The C2MSPK module specifically includes a first convolution and batch normalization submodule connected to the SPPF module, an N-layer MSPK (Multi-Scale Poly Kernel) submodule, a splicing submodule, and a second convolution and batch normalization submodule. The first output of the first convolution and batch normalization submodule is connected to the first layer MSPK submodule. The N layers of MSPK submodules are connected in series. The output of the last layer MSPK submodule is connected to the first input of the splicing submodule. The second output of the first convolution and batch normalization submodule is connected to the second input of the splicing submodule. The output of the splicing submodule is connected to the input of the second convolution and batch normalization submodule. The output of the second convolution and batch normalization submodule is connected to the upper application module and the splicing module of the neck network in the YOLOv11 structure.

[0007] The MSPK submodule includes an MSPK component, a RepCAA (Re-parameterization Context AnchorAttention) component, a first additive layer, and a first convolutional layer. The input ports of the MSPK submodule are connected to the MSPK component and the RepCAA component, respectively. The output ports of the MSPK component and the RepCAA component are both connected to the first additive layer. The output port of the first additive layer is connected to the first convolutional layer. The output port of the first convolutional layer serves as the output port of the MSPK submodule.

[0008] The MSPK component includes a 3×3 depth-separable convolutional layer, a 1×5 depth-separable convolutional layer, a 5×1 depth-separable convolutional layer, a 1×7 depth-separable convolutional layer, a 7×1 depth-separable convolutional layer, a 1×9 depth-separable convolutional layer, a 9×1 depth-separable convolutional layer, a 1×11 depth-separable convolutional layer, an 11×1 depth-separable convolutional layer, a second addition layer, a second convolutional layer, and a first multiplication layer. The output of the 3×3 depth-separable convolutional layer is connected to the 1×5 depth-separable convolutional layer, the 1×7 depth-separable convolutional layer, the 1×9 depth-separable convolutional layer, and the 1×11 depth-separable convolutional layer, respectively. The output of the 1×5 depth-separable convolutional layer is connected to the 5×1 depth-separable convolutional layer. The output of the 1×7 depth-separable convolutional layer is connected to the 7×1... A depthwise separable convolutional layer is included; the output of the 1×9 depthwise separable convolutional layer is connected to a 9×1 depthwise separable convolutional layer; the output of the 1×11 depthwise separable convolutional layer is connected to an 11×1 depthwise separable convolutional layer; the outputs of the 5×1, 7×1, 9×1, and 11×1 depthwise separable convolutional layers are connected to a second adder layer, which is also connected to the output of the 3×3 depthwise separable convolutional layer, and the output of the second adder layer is connected to the second convolutional layer; the output of the second convolutional layer is connected to the first multiplication layer, the input of the first multiplication layer is also connected to the input port of the MSPK component, and the output of the first multiplication layer serves as the output port of the MSPK component.

[0009] The RepCAA component includes an average pooling layer, a third convolutional layer, a 1×11 depthwise separable convolutional layer, an 11×1 depthwise separable convolutional layer, a fourth convolutional layer, a sigmoid activation function, and a second multiplication layer. The output of the average pooling layer is connected to the third convolutional layer. The output of the third convolutional layer is connected to the 1×11 depthwise separable convolutional layer and the 11×1 depthwise separable convolutional layer, respectively. The outputs of the 1×11 depthwise separable convolutional layer and the 11×1 depthwise separable convolutional layer are connected to the fourth convolutional layer. The output of the fourth convolutional layer is connected to the sigmoid activation function. The output of the sigmoid activation function is connected to the second multiplication layer. The input of the second multiplication layer is also connected to the input port of the RepCAA component. The output of the second multiplier serves as the output port of the RepCAA component.

[0010] The visual detection model for UAVs provided in this application adopts the YOLOv11 structure, which can accurately identify multiple targets in images or videos in real time, and classify and accurately locate the targets. The model has been improved, specifically by modifying its C2PSA module to a C2MSPK module. It utilizes depthwise separable convolutions to aggregate local information, then employs multi-branch depthwise separable convolutions to capture multi-scale context, and uses 1×1 convolutions to model the relationships between different channels. This module design not only improves model performance but also enhances the detection capability for small targets. Simultaneously, a reparameterized context anchor box attention component is used, changing the original serial structure to a parallel structure. This achieves the same effect during training, and during inference, the structure can be transformed into an 11×11 depthwise separable convolution using a structure reparameterization method. This approach maintains performance while reducing the model's computational complexity. The visual detection model for UAVs provided in this application improves the original model by introducing multi-scale attention and structure reparameterization, thereby enhancing the target detection task with a large scale span.

[0011] Preferably, the bounding box regression loss function of the head network of the visual detection model for the UAV is:

[0012] ;

[0013] in, ;in This indicates the percentage of intersection between the predicted bounding box and the ground truth bounding box. It is the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box. It is the diagonal length of the smallest enclosing rectangle. It is the difference in aspect ratio between the predicted bounding box and the ground truth bounding box. It is used for balance Weighted terms for aspect ratio;

[0014] ,in , All of them are real numbers in the range [0,1].

[0015] Preferably, the head network of the visual detection model for the UAV is used to determine whether fishing behavior exists. Specifically, it determines whether a person and a fishing rod exist. If they exist, it determines whether the person frame and the fishing rod frame overlap. If they do, it determines that fishing behavior exists and marks the person as having fishing behavior. For people and fishing rods whose person frames and fishing rod frames do not overlap, it constructs an N×M cost matrix cost, where N is the number of fishing rods and M is the number of people. It calculates the relative positional relationship of the closest points between the person frame and the fishing rod frame to obtain the normalized distance. It uses the Hungarian algorithm to perform optimal matching on the cost matrix to minimize the total distance. It matches each pair of person frames and fishing rod frames. If the normalized distance is less than or equal to a set threshold, the match is successful, and the corresponding person is determined to have fishing behavior and is marked. Otherwise, it is determined that fishing behavior does not exist.

[0016] This application also provides a fishing detection system, including a drone, on which the aforementioned visual detection model for drones is deployed. The main improvement of the fishing detection system provided in this application is that the processor deploys a trained and mature visual detection model for drones. Therefore, the fishing detection system provided in this application has the beneficial effects of a visual detection model for drones.

[0017] This application also provides a method for detecting fishing using a drone, including the following steps:

[0018] S100, Construct a visual inspection model for drones;

[0019] S200: Acquire fishing images of real-world scenes captured by drones;

[0020] S300. The fishing images are labeled, and then the labeled fishing data is divided into training set and test set;

[0021] S400. The visual detection model for the UAV is trained using the training set and the test set.

[0022] S500. After the visual detection model for the UAV has been trained and converged, the visual detection model for the UAV with the highest accuracy is deployed in the UAV's processor.

[0023] S600: Control the drone to perform inspections, and capture real-time video through the camera and input it into the visual detection model for the drone deployed in the drone processor to determine whether there are human figures and fishing rods. When human figures and fishing rods are determined to be present, determine whether there is fishing behavior. If so, mark it.

[0024] The fishing detection method for drones provided in this application adopts a visual detection model for drones based on an improved YOLOv11 architecture. The C2PSA module is improved to a C2MSPK module. Depthwise separable convolutions are used to aggregate local information, followed by multi-branch depthwise separable convolutions to capture multi-scale context, and 1×1 convolutions to model the relationships between different channels. This modular design not only improves model performance but also enhances the detection capability for small targets. Simultaneously, a reparameterized context anchor box attention component is adopted, changing the original serial structure to a parallel structure. This achieves the same effect during the training phase, and during the inference phase, the structure can be transformed into an 11×11 depthwise separable convolution using a structure reparameterization method. This approach maintains performance while reducing the computational complexity of the model. The visual detection model for drones provided in this application improves the original model by introducing multi-scale attention and structure reparameterization, enhancing the detection of targets with large scale spans. Furthermore, matching logic is added, matching the person with the fishing rod before reporting the message, significantly reducing the false alarm rate.

[0025] Preferably, step S600 specifically includes:

[0026] S610. Control the drone to perform inspections, and capture real-time video through the camera and input it into the visual detection model for the drone deployed in the drone processor to determine whether there is a human figure and a fishing rod. When it is determined that there is a human figure and a fishing rod, proceed to step S620.

[0027] The S620 drone hovers and zooms in for secondary confirmation, initiating the detection of fishing behavior.

[0028] S630. Determine whether the portrait frame and the fishing rod frame overlap. If they do, determine that there is fishing behavior and mark the portrait.

[0029] Preferably, after step S630, the method further includes:

[0030] S640. Determine if there are any unlabeled human figures and fishing rods. If so, construct an N×M cost matrix cost, where N is the number of fishing rods and M is the number of human figures.

[0031] S650. Calculate the relative positional relationship of the closest points between the portrait frame and the fishing rod frame to obtain the normalized distance; use the Hungarian algorithm to perform optimal matching on the cost matrix to minimize the total distance.

[0032] S660. Match each pair of portrait frames and fishing rod frames. If the normalized distance is less than or equal to the set threshold, the match is successful, and the corresponding portrait is determined to have fishing behavior and is marked. Otherwise, it is determined that there is no fishing behavior.

[0033] This application also provides an electronic device, including a memory and a processor. The memory stores a computer-executable program, and the processor calls the computer-executable program in the memory to implement the above-described method for detecting fishing using a drone.

[0034] This application embodiment also provides a storage medium, which is a computer-readable storage medium storing a computer-executable program. When the computer-executable program is executed by a processor, it implements the above-described fishing detection method for drones.

[0035] This application also provides a computer program product, including code that implements the above-described method for detecting fishing using unmanned aerial vehicles. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a schematic diagram of the YOLOV11 structure;

[0038] Figure 2 A schematic diagram of a visual inspection model for a drone provided in an embodiment of this application;

[0039] Figures 3A to 3D This is a schematic diagram of the C2MSPK module structure provided in an embodiment of this application;

[0040] Figures 4A to 4C This is a schematic diagram of the C3K2 module structure;

[0041] Figure 5 This is a schematic diagram of the SPPF module structure;

[0042] Figure 6 This is a schematic diagram of the Head detection head structure;

[0043] Figure 7A , Figure 7B Images of fishing in real-world settings were collected;

[0044] Figure 8 For data augmentation and synthetic datasets;

[0045] Figure 9 The image is labeled.

[0046] Figure 10A , Figure 10B This is a diagram illustrating the identified fishing behavior. Detailed Implementation

[0047] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0048] It is important to note that terms such as "first," "second," "symmetric," and "array" are used only to distinguish between descriptive and positional descriptions and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, features specified with terms such as "first" or "symmetric" may explicitly or implicitly include one or more of that feature; similarly, when the quantity of certain features is not limited by words such as "two" or "three," it should be noted that such features also explicitly or implicitly include one or more features.

[0049] In this invention, unless otherwise explicitly specified and limited, terms such as "installation," "connection," and "fixation" should be interpreted broadly; for example, they can refer to a fixed connection, a detachable connection, or an integral molding; they can refer to a mechanical connection, a direct connection, a welding connection, or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the accompanying drawings and specific circumstances.

[0050] The technical solution of this application will now be described in detail with reference to the accompanying drawings.

[0051] This application provides a fishing detection system that uses a drone to patrol relevant water areas such as riverbanks, lakesides, and canals, automatically identifying the presence of fishing activities. If fishing is detected, the system marks the area and sends images and alarm signals, thereby assisting management personnel in monitoring non-compliant fishing activities. The fishing detection system provided in this application includes a drone and a visual detection model for the drone deployed on the drone's processor. The drone provided in this application can be any currently available mature drone, preferably a quadcopter drone, a multi-rotor drone, or a vertical take-off and landing fixed-wing drone capable of hovering. The visual detection model for the drone is an improved version of YOLOv11, suitable for drones with large zoom capabilities and a wide range of target scales.

[0052] like Figure 1The diagram shows the original YOLO V11 structure. As the latest member of the YOLO series, YOLO V11 brings a series of improvements in image recognition and target detection, with more powerful feature extraction, more intelligent feature selection, and more accurate detection. In drone aerial image recognition, especially for feature recognition and classification of small targets and complex backgrounds, YOLO V11 improves detection sensitivity and accuracy while maintaining high speed.

[0053] However, when applying the YOLOv11 structural model to UAVs for fishing detection, the high-altitude shooting of the UAV results in a large zoom range for its camera, leading to a wide scale range for the same target. Simultaneously, it needs to detect and recognize both human figures and fishing rods, two objects with significantly different sizes. Existing YOLOv11 suffers from insufficient accuracy and slow computation speed when recognizing targets with such a large size range. Therefore, this application improves the YOLOv11 structural model by replacing its C2PSA module with a C2MSPK module to address the aforementioned problems encountered by existing UAVs in detecting, recognizing, and locating objects with large size differences.

[0054] like Figure 2 The diagram shows the structure after replacing the C2PSA module in YOLOv11 with the C2MSPK module in this application. The input of the C2MSPK module located in the Backbone network is connected to the SPPF module, and its output is connected to the upsampling module of the Neck network and the stitching module at the end, respectively. The specific structure of the C2MSPK module provided in this application embodiment is shown in Figure 3. The C2MSPK module specifically includes a first convolution and batch normalization processing submodule whose input is connected to the SPPF module, an N-layer MSPK (Multi-Scale Poly Kernel) submodule, a stitching submodule, and a second convolution and batch normalization processing submodule. In a preferred embodiment of this application, the MSPK submodule includes two layers. The first output of the first convolution and batch normalization submodule is connected to the first MSPK submodule. The first MSPK submodule and the second MSPK submodule are connected in series, that is, the output of the first MSPK submodule is connected to the second MSPK submodule. The output of the second MSPK submodule is connected to the first input of the splicing submodule. The second output of the first convolution and batch normalization submodule is connected to the second input of the splicing submodule. The output of the splicing submodule is connected to the input of the second convolution and batch normalization submodule. The output of the second convolution and batch normalization submodule is connected to the upper application module and the splicing module of the neck network in the YOLOv11 structure.

[0055] The MSPK submodule includes an MSPK component, a RepCAA (Re-parameterization Context AnchorAttention) component, a first additive layer, and a first convolutional layer. The input ports of the MSPK submodule are connected to the MSPK component and the RepCAA component, respectively. The output ports of the MSPK component and the RepCAA component are connected to the first additive layer. The output port of the first additive layer is connected to the first convolutional layer. The output port of the first convolutional layer serves as the output port of the MSPK submodule.

[0056] The MSPK component includes a 3×3 depth-separable convolutional layer, a 1×5 depth-separable convolutional layer, a 5×1 depth-separable convolutional layer, a 1×7 depth-separable convolutional layer, a 7×1 depth-separable convolutional layer, a 1×9 depth-separable convolutional layer, a 9×1 depth-separable convolutional layer, a 1×11 depth-separable convolutional layer, an 11×1 depth-separable convolutional layer, a second addition layer, a second convolutional layer, and a first multiplication layer. The output of the 3×3 depth-separable convolutional layer is connected to the 1×5 depth-separable convolutional layer, the 1×7 depth-separable convolutional layer, the 1×9 depth-separable convolutional layer, and the 1×11 depth-separable convolutional layer, respectively. The output of the 1×5 depth-separable convolutional layer is connected to the 5×1 depth-separable convolutional layer. The output of the 1×7 depth-separable convolutional layer is connected to the 7×1... A depthwise separable convolutional layer is provided; the output of the 1×9 depthwise separable convolutional layer is connected to a 9×1 depthwise separable convolutional layer; the output of the 1×11 depthwise separable convolutional layer is connected to an 11×1 depthwise separable convolutional layer; the outputs of the 5×1, 7×1, 9×1, and 11×1 depthwise separable convolutional layers are connected to a second adder layer, which is also connected to the output of the 3×3 depthwise separable convolutional layer, and the output of the second adder layer is connected to the second convolutional layer; the output of the second convolutional layer is connected to the first multiplication layer, the input of the first multiplication layer is also connected to the input port of the MSPK component, and the output of the first multiplication layer serves as the output port of the MSPK component.

[0057] The RepCAA component includes an average pooling layer, a third convolutional layer, a 1×11 depthwise separable convolutional layer, an 11×1 depthwise separable convolutional layer, a fourth convolutional layer, a sigmoid activation function, and a second multiplication layer. The output of the average pooling layer is connected to the third convolutional layer. The output of the third convolutional layer is connected to the 1×11 depthwise separable convolutional layer and the 11×1 depthwise separable convolutional layer, respectively. The outputs of the 1×11 depthwise separable convolutional layer and the 11×1 depthwise separable convolutional layer are connected to the fourth convolutional layer. The output of the fourth convolutional layer is connected to the sigmoid activation function. The output of the sigmoid activation function is connected to the second multiplication layer. The input of the second multiplication layer is also connected to the input port of the RepCAA component. The output of the second multiplier serves as the output port of the RepCAA component.

[0058] like Figure 2 As shown, the visual detection model for UAVs provided in this application first processes the acquired images or videos through first and second convolutional blocks. Each convolutional block consists of a regular convolution with a kernel of 3×3, a stride of 2, and padding of 1, a batch normalization layer, and a SiLU activation function. Then, it is processed by the first C3K2 module, whose network structure is as follows: Figure 4A As shown in the diagram, the input to the C3K2 module is first processed by a 1×1 convolutional block, which consists of a regular convolution with a 1×1 kernel and a stride of 1, a batch normalization layer, and a SiLU activation function. Then, a channel splitting operation is performed, with one portion directly input to the stitching layer and the other portion both directly input to the stitching layer and processed through two C3K modules. The result of the first C3K module also participates in the stitching operation. Finally, the stitched result is processed by another 1×1 convolutional block to obtain the final output. The structure of the C3K module is as follows: Figure 4B As shown, the input is processed by two 1×1 convolutional blocks, left and right. The output of the left 1×1 convolutional block is processed through two bottleneck modules before being input to the concatenation layer. The output of the right 1×1 convolutional block is directly input to the concatenation layer. The concatenated output is then processed by a second 1×1 convolutional block to obtain the final output. The structure of the bottleneck module is as follows: Figure 4C As shown, the input is first processed through two 1×1 convolutional blocks, and then the output is obtained after being added together by jump connections.

[0059] The output of the first C3K2 module is processed by the third 3×3 convolutional block and then input to the second C3K2 module, then to the third convolutional block, then processed by the third C3K2 module, and finally input to the SPPF layer. The structure of the SPPF layer is as follows: Figure 5 As shown.

[0060] The SPPF input is first processed through a 1×1 convolutional block, which contains a 1×1 regular convolution, a batch normalization layer, and a SiLU activation function layer. The output of the 1×1 convolutional block is either directly input to the concatenation layer or passes through three max pooling layers with a kernel size of 5×5. The output of each pooling layer is then input to the concatenation layer, and the output of the concatenation layer passes through another 1×1 convolutional block to obtain the final output. The output of the SPPF layer is then input to the C2MSPK module for processing.

[0061] As shown in Figure 3, the input to the C2MSPK module first undergoes a 1×1 convolution and batch normalization. The processed structure is then processed through N layers of MSPK sub-modules (N is 2 in this invention). After processing, the output is concatenated with the output of the first convolutional layer. The concatenated output is then fed into a second 1×1 convolution and batch normalization to obtain the final output. The MSPK module input is processed by both the MSPK and RepCAA components, and the outputs of the two components are added together before undergoing a 1×1 convolution to obtain the final output. The MSPK component input first undergoes a depthwise separable convolution with a 3×3 kernel. The processed result is then processed by depthwise separable convolutions of 1×5 and 5×1, 1×7 and 7×1, 1×9 and 9×1, and 1×11 and 11×1. Larger kernels are used here because they ensure a larger receptive field, maximizing the extraction of contextual features. The structure with all large convolutional kernels is added to the structure of the first 3×3 depthwise separable convolution. After addition, it is processed by a 1×1 convolution. The result is then multiplied by the input to obtain the final output. The input of the RepCAA component first passes through an average pooling layer, then a 1×1 convolution, followed by a 1×11 depthwise separable convolution to extract horizontal features and an 11×1 depthwise separable convolution to extract vertical features. During the inference phase, this structure can be reparameterized into an 11×11 depthwise separable convolution. Then, it passes through a 1×1 convolution, followed by a sigmoid activation function. Finally, the output of the sigmoid activation function is multiplied by the input to obtain the final output.

[0062] like Figure 2As shown, the signal output from the C2MSPK module passes through the first upsampling layer. The output of the first upsampling layer is concatenated with the output of the second C3K2 module. After concatenation, it is processed by the fourth C3K2 module and then input to the second upsampling layer. It is then concatenated with the output of the first C3K2 module and input to the fifth C3K2 layer. The output of the fifth C3K2 layer is directly input to the first detection head and simultaneously input to the fifth convolution block. After processing, it is concatenated with the output of the fourth C3K2 module and input to the sixth C3K2 module. The output of the sixth C3K2 module is input to the second detection head and simultaneously input to the sixth convolution block. The output of the sixth 3×3 convolution block is concatenated with the output of the C2MSPK module and then input to the seventh C3K2 module for processing. After processing, it is input to the third detection head.

[0063] The structure of the detection head is as follows Figure 6 As shown, the input to the detection head is processed on both the left and right sides. The left structure first goes through two 3×3 convolutional blocks, then through a 1×1 convolution with a stride of 1 and padding of 0 to obtain the final output. The right structure first goes through a depthwise separable convolution with a stride of 1 and padding of 1 and a kernel of 3×3, then through a 3×3 convolutional block, and then through a second depthwise separable convolution with a stride of 1 and padding of 1 and a kernel of 3×3. This is followed by a second 3×3 convolutional block, and finally through a 1×1 convolution with a stride of 1 and padding of 0 to obtain the final output.

[0064] The visual detection model for UAVs provided in this application adopts the YOLOv11 structure, which can accurately identify multiple targets in images or videos in real time, and classify and accurately locate the targets. The model has been improved, specifically by modifying its C2PSA module to a C2MSPK module. It utilizes depthwise separable convolutions to aggregate local information, then employs multi-branch depthwise separable convolutions to capture multi-scale context, and uses 1×1 convolutions to model the relationships between different channels. This module design not only improves model performance but also enhances the detection capability for small targets. Simultaneously, a reparameterized context anchor box attention component is used, changing the original serial structure to a parallel structure. This achieves the same effect during training, and during inference, the structure can be transformed into an 11×11 depthwise separable convolution using a structure reparameterization method. This approach maintains performance while reducing the model's computational complexity. The visual detection model for UAVs provided in this application improves the original model by introducing multi-scale attention and structure reparameterization, thereby enhancing the target detection task with a large scale span.

[0065] During the careful training of the visual detection model for UAVs provided in this application, the detection head network mainly includes bounding box regression loss, classification loss, and distribution loss. The model parameters are adjusted through backpropagation by calculating the error between the detection head network output and the true value. This invention improves the bounding box loss by introducing Focaler-CIoU loss. The original bounding box regression loss used CIoU, and its mathematical formula is as follows:

[0066] ;

[0067] The improved Focaler-CIoU mathematical formula is as follows:

[0068] ;

[0069] in The mathematical formula is as follows:

[0070] ;

[0071] in , All are real numbers in the range [0,1]. By adjusting... , The value of d allows IoU to focus on different regression samples. In this invention, d is set to 0.1 and u is set to 0.9.

[0072] Where IoU represents the percentage of intersection between the predicted bounding box and the ground truth bounding box. It is the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box. It is the diagonal length of the smallest enclosing rectangle. It is the difference in aspect ratio between the predicted bounding box and the ground truth bounding box. It is a weighting term used to balance IoU and aspect ratio.

[0073] The classification loss uses the cross-entropy loss function, whose mathematical formula is as follows:

[0074]

[0075] Where N is the number of targets and M represents the number of categories. The sign function (0 or 1) if the sample If the category is equal to c, then take 1; otherwise, take 0. Observational Samples The predicted probability of belonging to category c.

[0076] The distributed loss uses the DFL loss, and its mathematical formula is as follows:

[0077]

[0078] in Let represent the predicted probabilities of the two points to the left and right of the true point y, respectively. , This represents the coordinates of the two points to the left and right of the actual point.

[0079] During training, the model is trained using all three loss functions mentioned above. After the model converges, the model with the highest accuracy is deployed to the drone's processor. In use, real-time video stream data captured by the drone is input into the algorithm model. The algorithm model identifies whether there are people or fishing rods in the footage. If so, it determines whether fishing activity exists and labels the fishing activity. The labeling results are sent to the corresponding receiving terminal for easy viewing and management by administrators.

[0080] Based on the above-described fishing detection system, this application provides a fishing detection method, which specifically includes the following steps:

[0081] S100. Construct a visual detection model for the UAV. The specific structure of the visual detection model for the UAV in this embodiment can be found in the preceding description and will not be repeated here.

[0082] S200: Acquire fishing images from real-world scenes captured by drones.

[0083] In this embodiment, a drone can be controlled to photograph water bodies such as riverbanks, lakeshores, etc., to obtain fishing images under different perspectives, shooting distances, fractal heights, and lighting conditions in real-world scenarios. Simultaneously, to collect data more quickly and in greater quantities, fishing images meeting the requirements can also be collected through online channels. The collected fishing images can be found in [reference needed]. Figure 7A , Figure 7B .

[0084] After collecting fishing images from real-world scenarios that meet the requirements, the dataset can be expanded through data augmentation. First, the real-world images are cropped. Considering the low resolution of the human body and fishing rod from the drone's perspective, random scaling is applied. After scaling, a Poisson fusion is performed, and the synthesized data samples are shown below. Figure 8 As shown.

[0085] S300. The fishing images are labeled, and then the labeled fishing data is divided into training set and test set.

[0086] Data annotation was performed on all images using the X-anylabeling software, with the annotation format following the YOLO object detection standard. The annotation process is as follows: Figure 9 As shown.

[0087] S400. The visual detection model for the UAV is trained using the training set and the test set.

[0088] S500. After the visual detection model for the UAV has been trained and converged, the visual detection model with the highest accuracy for the UAV is deployed in the UAV's processor.

[0089] S600: Control the drone to perform inspections, and capture real-time video through the camera and input it into the visual detection model for the drone deployed in the drone processor to determine whether a human figure and fishing rod are present. When a human figure and fishing rod are detected, determine whether fishing behavior exists. If so, mark it. Step S600 specifically includes:

[0090] S610. Control the drone to perform inspections, and capture real-time video through the camera and input it into the visual detection model for the drone deployed in the drone processor to determine whether there is a human figure and a fishing rod. When it is determined that there is a human figure and a fishing rod, proceed to step S620.

[0091] The S620 drone hovers and zooms in for secondary confirmation, initiating the detection of fishing behavior.

[0092] S630. Determine whether the portrait frame and the fishing rod frame overlap. If they do, determine that there is fishing behavior and mark the portrait.

[0093] S640. Determine if there are any unlabeled human figures and fishing rods. If so, construct an N×M cost matrix cost, where N is the number of fishing rods and M is the number of human figures.

[0094] S650. Calculate the relative positional relationship between the closest points in the person frame and the fishing rod frame to obtain the normalized distance. Specifically, this is done by calculating the relative positional relationship between the closest points in the person frame and the fishing rod frame using a function (considering different camera focal lengths, pixel distance cannot be directly used, so the distance is normalized using the width of the person detection box as the denominator). Then, the Hungarian algorithm is used to perform optimal matching on the cost matrix to minimize the total distance.

[0095] S660. Match each pair of portrait frames and fishing rod frames. If the normalized distance is less than or equal to a set threshold, the match is successful, and the corresponding portrait is determined to have fishing behavior and is marked. Otherwise, it is determined that there is no fishing behavior. In this embodiment, the preset value is 1. The recognition effect is as follows: Figure 10A , Figure 10B .

[0096] The fishing detection method provided in this application utilizes deep learning algorithms to detect wild fishing based on drone footage. It also improves the original model by introducing multi-scale attention and structural reparameterization, thus enhancing the detection capabilities for targets with large scale spans. Furthermore, it performs matching calculations between the person's image and the fishing rod, ensuring that a match is made before reporting the information, significantly reducing the false alarm rate.

[0097] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in the present invention, and these should all be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for constructing a visual inspection model for unmanned aerial vehicles (UAVs), characterized in that, The visual detection model for drones is based on the YOLOv11 architecture, and the C2PSA module of the YOLOv11 architecture is replaced with the C2MSPK module; YOLOv11 takes drone aerial images as input and identifies whether there are human figures or fishing rods in the image; The C2MSPK module specifically includes a first convolution and batch normalization submodule connected to the SPPF module, an N-layer MSPK submodule, a splicing submodule, and a second convolution and batch normalization submodule. The first output of the first convolution and batch normalization submodule is connected to the first layer MSPK submodule. The N layers of MSPK submodules are connected in series. The output of the last layer MSPK submodule is connected to the first input of the splicing submodule. The second output of the first convolution and batch normalization submodule is connected to the second input of the splicing submodule. The output of the splicing submodule is connected to the input of the second convolution and batch normalization submodule. The output of the second convolution and batch normalization submodule is connected to the upper application module and the splicing module of the neck network in the YOLOv11 structure. The MSPK submodule includes an MSPK component, a RepCAA component, a first addition layer, and a first convolutional layer. The input ports of the MSPK submodule are connected to the MSPK component and the RepCAA component, respectively. The output ports of the MSPK component and the RepCAA component are both connected to the first addition layer. The output port of the first addition layer is connected to the first convolutional layer. The output port of the first convolutional layer serves as the output port of the MSPK submodule. The MSPK component includes a 3×3 depth-separable convolutional layer, a 1×5 depth-separable convolutional layer, a 5×1 depth-separable convolutional layer, a 1×7 depth-separable convolutional layer, a 7×1 depth-separable convolutional layer, a 1×9 depth-separable convolutional layer, a 9×1 depth-separable convolutional layer, a 1×11 depth-separable convolutional layer, an 11×1 depth-separable convolutional layer, a second addition layer, a second convolutional layer, and a first multiplication layer; the output of the 3×3 depth-separable convolutional layer is connected to the 1×5 depth-separable convolutional layer, the 1×7 depth-separable convolutional layer, the 1×9 depth-separable convolutional layer, and the 1×11 depth-separable convolutional layer, respectively. The output of the 1×5 depth separable convolutional layer is connected to a 5×1 depth separable convolutional layer; the output of the 1×7 depth separable convolutional layer is connected to a 7×1 depth separable convolutional layer. The output of the 1×9 depth-separable convolutional layer is connected to the 9×1 depth-separable convolutional layer; the output of the 1×11 depth-separable convolutional layer is connected to the 11×1 depth-separable convolutional layer; the outputs of the 5×1, 7×1, 9×1, and 11×1 depth-separable convolutional layers are connected to the second adder layer, which is also connected to the output of the 3×3 depth-separable convolutional layer, and the output of the second adder layer is connected to the second convolutional layer; the output of the second convolutional layer is connected to the first multiplication layer, the input of the first multiplication layer is also connected to the input port of the MSPK component, and the output of the first multiplication layer serves as the output port of the MSPK component; The RepCAA component includes an average pooling layer, a third convolutional layer, a 1×11 depthwise separable convolutional layer, an 11×1 depthwise separable convolutional layer, a fourth convolutional layer, a sigmoid activation function, and a second multiplication layer. The output of the average pooling layer is connected to the third convolutional layer. The output of the third convolutional layer is connected to the 1×11 depthwise separable convolutional layer and the 11×1 depthwise separable convolutional layer, respectively. The outputs of the 1×11 depthwise separable convolutional layer and the 11×1 depthwise separable convolutional layer are connected to the fourth convolutional layer. The output of the fourth convolutional layer is connected to the sigmoid activation function. The output of the sigmoid activation function is connected to the second multiplication layer. The input of the second multiplication layer is also connected to the input port of the RepCAA component. The output of the second multiplication layer serves as the output port of the RepCAA component.

2. The method for constructing a visual inspection model for unmanned aerial vehicles as described in claim 1, characterized in that, The bounding box regression loss function of the head network in the visual detection model for UAVs is: ； in, ;in This indicates the percentage of intersection between the predicted bounding box and the ground truth bounding box. It is the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box. It is the diagonal length of the smallest enclosing rectangle. It is the difference in aspect ratio between the predicted bounding box and the ground truth bounding box. It is used for balance Weighted terms for aspect ratio; ,in , All of them are real numbers in the range [0,1].

3. The method for constructing a visual inspection model for unmanned aerial vehicles as described in claim 2, characterized in that, The head network of the visual detection model for drones is used to determine whether fishing behavior exists. Specifically, it determines whether a human figure and a fishing rod exist. If they exist, it determines whether the human figure frame and the fishing rod frame overlap. If they do, it determines that fishing behavior exists and marks the human figure as a fishing behavior. For portraits and fishing rods whose frames do not overlap, construct an N×M cost matrix cost, where N is the number of fishing rods and M is the number of portraits. Calculate the relative positional relationship between the closest points between the person frame and the fishing rod frame to obtain the normalized distance; use the Hungarian algorithm to perform optimal matching on the cost matrix to minimize the total distance; For each pair of human portrait frames and fishing rod frames, a match is performed. If the normalized distance is less than or equal to a set threshold, the match is successful, and the corresponding human portrait is determined to have fishing behavior and is marked. Otherwise, it is determined that there is no fishing behavior.

4. A method for detecting fishing using unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: S100. Construct the visual detection model for UAV as described in claim 1; S200: Acquire fishing images of real-world scenes captured by drones; S300. The fishing images are labeled, and then the labeled fishing data is divided into training set and test set; S400. The visual detection model for the UAV is trained using the training set and the test set. S500. After the visual detection model for the UAV has been trained and converged, the visual detection model for the UAV with the highest accuracy is deployed in the UAV's processor. S600: Control the drone to perform inspections, and capture real-time video through the camera and input it into the visual detection model for the drone deployed in the drone processor to determine whether there are human figures and fishing rods. When human figures and fishing rods are determined to be present, determine whether there is fishing behavior. If so, mark it.

5. The method for detecting fishing using a drone as described in claim 4, characterized in that, Step S600 specifically includes: S610. Control the drone to perform inspections, and capture real-time video through the camera and input it into the visual detection model for the drone deployed in the drone processor to determine whether there is a human figure and a fishing rod. When it is determined that there is a human figure and a fishing rod, proceed to step S620. The S620 drone hovers and zooms in for secondary confirmation, initiating the detection of fishing behavior. S630. Determine whether the portrait frame and the fishing rod frame overlap. If they do, determine that there is fishing behavior and mark the portrait.

6. The method for detecting fishing using a drone as described in claim 5, characterized in that, After step S630, the method further includes: S640. Determine if there are any unlabeled human figures and fishing rods. If so, construct an N×M cost matrix cost, where N is the number of fishing rods and M is the number of human figures. S650. Calculate the relative positional relationship of the closest points between the portrait frame and the fishing rod frame to obtain the normalized distance; use the Hungarian algorithm to perform optimal matching on the cost matrix to minimize the total distance. S660. Match each pair of portrait frames and fishing rod frames. If the normalized distance is less than or equal to the set threshold, the match is successful, and the corresponding portrait is determined to have fishing behavior and is marked. Otherwise, it is determined that there is no fishing behavior.

7. A fishing detection system, characterized in that, The invention includes a drone, wherein the drone processor is equipped with a visual detection model for the drone as described in any one of claims 1 to 3.

8. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer-executable program, and the processor invokes the computer-executable program in the memory to implement the fishing detection method for unmanned aerial vehicles as described in any one of claims 4 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, on which a computer-executable program is stored, and when the computer-executable program is executed by a processor, it implements the fishing detection method for unmanned aerial vehicles as described in any one of claims 4 to 6.

10. A computer program product, characterized in that, Includes code for implementing the fishing detection method for drones as described in any one of claims 4 to 6.

Images