A drone-based intelligent inspection and image recognition system for fisheries patrols in large reservoirs.
By designing an intelligent drone inspection and image recognition system, automated image acquisition and target recognition for fishery patrols in large reservoirs have been achieved, generating coordinate watermarked evidence. This solves the problem of insufficient automatic recognition adaptability in existing technologies and improves patrol efficiency and accuracy.
Patent Information
- Application Number
- CN202611284882.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-24
- Publication Date
- 2026-11-10
AI Technical Summary
Existing technologies have not yet formed a complete intelligent inspection solution adapted to fishery patrols in large water reservoirs. In the scenario of fishery patrols in reservoirs, the size of the vessels is much smaller than that of fishing vessels operating at sea. Water surface reflections and shadows from trees on the shoreline significantly interfere with image acquisition. Existing systems mostly rely on manual review of the inspection footage frame by frame, and the automatic identification coverage of the core targets of fishery patrols is insufficient.
Design an intelligent drone inspection and image recognition system for fisheries patrol in large reservoirs. The system employs a drone terminal, a ground control terminal, and a back-end image recognition processing terminal to achieve automatic flight along a preset route and full image acquisition during the inspection process. It automatically marks the location information of suspicious vessels, illegal fishing gear, and illegal fishing personnel, and generates exportable coordinate watermarked evidence image files and inspection task reports, reducing the workload of manual screening.
It improves the operational efficiency of fishery patrols in large reservoirs, reduces the workload of manually screening images frame by frame, realizes automated target recognition and report generation, adapts to recognition under different lighting conditions, and reduces the problems of missed detection and fatigue.
Smart Images

Figure CN123071871A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of drone inspection and image recognition applications, specifically to an intelligent drone inspection and image recognition system for fisheries patrols in large water reservoirs. Background Technology
[0002] Traditional fisheries patrols of large reservoirs mainly rely on manual patrols by boat, which have limited coverage, making it difficult to reach remote bays and shallow areas. Night patrols also suffer from limited visibility, high manpower input but low efficiency, and delayed evidence collection of illegal fishing activities. Several related inspection technologies have been publicly disclosed. For example, the South China Sea Fisheries Research Institute of the Chinese Academy of Fishery Sciences has filed a patent (publication number CN118279773B): a method and system for monitoring prohibited fishing gear based on drones, mainly targeting the identification of the distribution of prohibited fishing gear on fishing vessels in nearshore waters. Guangdong Puhe Testing Technology Co., Ltd. has filed a patent (publication number CN119295983A): a method and system for intelligent reservoir safety inspection based on drones, focusing on monitoring reservoir dam structural anomalies and water level status. Nanjing Tuoheng Unmanned Systems Research Institute Co., Ltd. has filed a patent (publication number CN118894250A): a camera drone for reservoir quality inspection and its usage method, acquiring reservoir area topographic image data through a camera device. The paper "Application of Video Image Target Detection Technology in Reservoir Area Inspection" by Wang Junwen et al. from the Water Transport Research Institute of the Ministry of Transport, published in the 11th issue of *Modern Information Technology* in 2020, proposes using video image target detection technology to collect evidence of illegal fishing in the reservoir area of a navigation and hydropower hub. The paper "Research on UAV-based River Floating Object Inspection Technology" published in the 5th issue of *Engineering Survey*, a journal sponsored by the China Water Resources Society, in 2018, outlines the human-machine interactive recognition process of UAV images of floating objects on the river surface. Current technologies have not yet formed a complete intelligent inspection solution fully adapted to the fisheries scenario of large-scale reservoirs. In reservoir fisheries inspection scenarios, the size of vessels is much smaller than that of fishing vessels operating at sea, and water surface reflections and shadows from trees along the shoreline significantly interfere with image acquisition. Existing systems mostly adopt a manual frame-by-frame review mode for inspection footage, resulting in insufficient adaptability of automatic identification to cover the core targets of fisheries inspections. Summary of the Invention
[0003] The purpose of this invention is to overcome the shortcomings of existing technologies and provide an intelligent drone inspection and image recognition system for fisheries patrols in large reservoirs. This system enables automatic flight along preset routes and full image acquisition and storage throughout the inspection process. The acquired images are compared with pre-stored fisheries patrol target sample data, automatically marking the location information of suspicious vessels, illegal fishing gear, and illegal fishing personnel appearing in the water. It directly generates exportable image files with coordinate watermarks and inspection task reports, reducing the workload of manual frame-by-frame image screening and improving the operational efficiency of daily fisheries patrols in large reservoirs.
[0004] To achieve the above objectives, the technical solution of this invention is to design an intelligent drone inspection and image recognition system for fishery patrols in large-scale reservoirs. The system consists of a drone terminal, a ground control terminal, and a back-end image recognition processing terminal. The ground control terminal comprises a drone flight controller and a touchscreen. The ground control terminal is signal-connected to the drone terminal. The back-end image recognition processing terminal receives photos taken by the drone terminal, processes them, and generates an inspection report. The flight platform has a maximum effective payload of no less than 1.5 kg, a single battery runtime of no less than 30 minutes, a camera imaging resolution of no less than 1920×1080 pixels, and a camera pitch angle adjustable within a range of ±90 degrees. An onboard positioning module records the drone's coordinate data in real time. The operator marks waypoints on an electronic map loaded on the touchscreen interface to generate a reservoir inspection route. After issuing a flight mission command with one click, the drone automatically flies according to the waypoint sequence. During flight, the touchscreen displays the drone's current position, remaining battery power, and real-time images transmitted from the camera.
[0005] A further technical solution involves an unmanned aerial vehicle (UAV) equipped with a multi-rotor flight platform featuring at least one visible light gimbal camera and an onboard positioning module. The backend image recognition processing unit, deployed on a local computer, includes a fisheries patrol target image sample library and a task data storage module. During the UAV's flight along a preset route, the gimbal camera automatically captures photos at preset time intervals, overlays coordinate watermarks, and transmits them to the backend storage. The backend image recognition processing unit divides the image into multiple detection windows, extracts color feature vectors from each window, and calculates the similarity with the feature vectors of each sample in the sample library. Detection windows with similarity exceeding a preset threshold are identified as suspected target areas, and their coordinate information is recorded. After recognition, an inspection report containing the coordinates of the suspected target's location and the image file is generated. Using the proportion of pixels in each color channel as the feature vector, rather than absolute color values, allows the feature extraction to adapt to brightness variations under different lighting conditions, reducing interference from water reflections and shoreline tree shadows on the recognition results.
[0006] A further technical solution is that the color feature vector is extracted by counting the number of pixels in each of the three RGB color channels in the image area within four intervals: 0 to 63, 64 to 127, 128 to 191, and 192 to 255. The number of pixels in each interval is divided by the total number of pixels in the area to obtain 12 ratio values to form the feature vector.
[0007] A further technical solution is to calculate the similarity by subtracting the 12 values at the corresponding positions of the detection window feature vector and the sample feature vector, taking the absolute values, summing them, dividing the sum by 12 to obtain the average difference value, and subtracting the average difference value from 1 to obtain the similarity score.
[0008] A further technical solution is to make the detection window square, with equal width and height, and set within the range of 200 pixels to 500 pixels. Extraction is performed from left to right and from top to bottom with a fixed step size, and the step size is set to half the side length of the detection window so that adjacent windows overlap by half.
[0009] A further technical solution is that the drone's flight parameters include flight altitude, flight speed, and shooting interval. The system has a built-in flight parameter configuration table that lists recommended flight speeds and shooting intervals for different flight altitudes for the operator to select.
[0010] A further technical solution involves setting a preset threshold that can be adjusted by the operator through the system configuration interface. The default value is set to 0.6, and the value range is between 0.4 and 0.9. The sample library index file uses text format, with each line recording information for one sample, including a category label and 12 feature values. Multiple samples are appended sequentially to form a complete sample library. The preset threshold can be adjusted by the operator through the system configuration interface. The default value is set to 0.6, and the value range is between 0.4 and 0.9. Higher values result in stricter identification, increasing the false negative rate and decreasing the false positive rate; lower values result in more lenient identification, decreasing the false negative rate and increasing the false positive rate.
[0011] A further technical solution is that the coordinate watermark position can be selected as the upper left, upper right, lower left, or lower right corner in the system configuration interface; the backend image recognition processing terminal adopts a multi-threaded parallel processing method to process multiple images simultaneously to improve recognition efficiency; the inspection report is exported in PDF format, including the file name, shooting coordinates, matched sample category label, and similarity score of each suspected target area image. After the UAV takes off from the ground hangar or take-off and landing point, it flies point by point along the preset route. The system has a built-in flight parameter configuration table, which lists the recommended flight speed and shooting interval corresponding to different flight altitudes. A flight altitude of 80 meters corresponds to a flight speed of 2.5 meters per second and a shooting interval of 2.5 seconds; a flight altitude of 100 meters corresponds to a flight speed of 3 meters per second and a shooting interval of 3 seconds; a flight altitude of 120 meters corresponds to a flight speed of 3.5 meters per second and a shooting interval of 3.5 seconds; a flight altitude of 150 meters corresponds to a flight speed of 4.5 meters per second and a shooting interval of 4.5 seconds; and a flight altitude of 200 meters corresponds to a flight speed of 6 meters per second and a shooting interval of 6 seconds. The operator can select any configuration in the table or make minor adjustments based on the values in the configuration table according to actual needs. After each photo is taken, the system automatically reads the latitude, longitude, and altitude values currently recorded by the onboard positioning module, and overlays these values as white text on the edge of the photo to create a photo file with coordinate watermarks.
[0012] After all images have been identified, the system automatically generates a list of identification results. Operators can view the location markers of all suspected target points on the electronic map interface. Clicking on any marker will bring up a corresponding image thumbnail. Images not identified as suspected targets are archived in the historical database in chronological order of capture time for later retrieval.
[0013] The advantages and beneficial effects of this invention are as follows: all components of this system can be assembled using commercially available standard components to achieve a fully visualized operation interface throughout the entire process. Operators do not need to have professional algorithm knowledge; they can directly follow the interface guidance to complete the route import task and carry out daily patrols. All image files collected and generated by the system retain the original shooting parameters and cannot be tampered with, and can be directly used as archival evidence materials for fishery law enforcement. Attached Figure Description
[0014] Figure 1 This is a schematic diagram of a first embodiment of the intelligent inspection and image recognition system for fisheries patrol of large water reservoirs according to the present invention; Figure 2 This is a schematic diagram of Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the mechanical tilt recording device below the drone in Embodiment 4 of the present invention; Figure 4 This is the data processing workflow of Embodiment 4 of the present invention; Figure 5 This is a record of the fluctuation of the composite tilt angle evaluation value under a level 4 gust in Embodiment 4 of the present invention; Figure 6 This is a schematic diagram of the structure of the UAV in Embodiment 5 of the present invention; Figure 7 yes Figure 6 A schematic diagram showing the ground coverage area captured by a drone flying at a height of 120 meters; Figure 8 This is a flowchart of the multi-view differential preprocessing in Embodiment 5 of the present invention.
[0015] In the diagram: 1. Counterweight; 2. Swing arm; 3. Universal bearing assembly; 4. Photoelectric angle encoder; 5. Drone; 6. Gimbal camera; 7. Auxiliary camera; 8. Support. Detailed Implementation
[0016] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and examples. The following examples are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.
[0017] Example 1: As Figure 1As shown, this invention is an intelligent drone inspection and image recognition system for fishery patrols in large reservoirs. It includes a drone terminal, a ground control terminal, and a back-end image recognition processing terminal. The drone terminal is equipped with a multi-rotor flight platform with a visible light gimbal camera and an onboard positioning module. The ground control terminal includes a drone flight controller and a touch screen. The back-end image recognition processing terminal is deployed in a local computer and includes a fishery patrol target image sample library and a task data storage module.
[0018] A medium-sized reservoir has a total area of 12 square kilometers and a total shoreline of 28 kilometers. Routine fisheries patrols utilize this system, equipped with a quadcopter drone platform and a 1 / 2.3-inch visible light gimbal camera. The onboard positioning module operates in dual-mode, integrating both BeiDou and GPS satellite positioning signals. The ground control unit uses a 10-inch touchscreen tablet with a built-in high-definition electronic map of the reservoir area. On the map interface, the operator sequentially clicks to set 128 waypoints along the main reservoir channel, three remote bays, and the surrounding shoreline, generating a closed patrol route of 32 kilometers. In the flight parameter configuration table, the operator selects the configuration group corresponding to a flight altitude of 120 meters, a flight speed of 3.5 meters per second, a shooting interval of 3.5 seconds, sets the gimbal camera to vertical downward shooting, and selects the lower right corner for the coordinate watermark. With a single click to start the patrol mission, the drone automatically takes off from the shore landing point and flies sequentially along the preset waypoints.
[0019] The entire flight took approximately 150 minutes, with two automatic return trips to replace the batteries. The onboard camera captured a total of 2,600 high-definition images, all of which were continuously transmitted wirelessly to the local hard drive of the image recognition processing unit. A sample database was established in advance. Fisheries officials collected 10 photos each of five common types of small fishing boats, 8 photos each of four types of illegal gillnets, and 6 photos each of two types of electrofishing equipment. The operator opened each photo, selected the target area on the screen using the mouse, and entered category labels such as Type A fishing boat, Type B gillnet, etc. After saving, the system automatically extracted the color feature vector of the area and wrote it into the sample database index file. The final sample database contained 94 sample records, each with a category label and 12 feature values.
[0020] After the flight mission concludes, the operator clicks the "Start Recognition" button. The backend processing module begins batch reading all 2600 image files from the mission. Each image has a resolution of 5472×3648 pixels. The operator sets the detection window size to 350×350 pixels and the step size to 175 pixels in the system configuration interface. Approximately 620 detection windows are extracted from each image based on the window size and step size. Color feature vectors are extracted from each detection window, and a similarity score is obtained by calculating the average absolute error with 94 sample records in the sample library. The highest score is selected, with a preset threshold of 0.6. When the similarity score between a detection window and a sample in the sample library exceeds 0.6, the detection window is marked as a suspected target. The backend processing unit uses an industrial PC equipped with an Intel Core i7 eight-core processor with a clock speed of 3.2GHz and 16GB of memory. It runs a recognition program developed using Python and the OpenCV open-source image processing library. The program uses multi-threaded parallel processing to process eight images simultaneously. The entire batch recognition process takes several hours depending on the hardware performance.
[0021] During the identification process, a detection window in image number 1247 detected a suspected target with a similarity score of 0.72 to a Type A fishing vessel in the sample database, exceeding the threshold. The system automatically displayed the image and marked its latitude and longitude coordinates on the electronic map. The operator confirmed that the image indeed showed a small fishing vessel with a hull color similar to Type A fishing vessels in the sample database. After confirming, the system copied the image to the evidence folder and added a record to the inspection report containing the image number, coordinates, matching sample category, and similarity score. The identification process continued. In image number 1803, a suspected target with a similarity score of 0.68 to a Type B wire mesh sample was detected. In image number 2456, a suspected target with a similarity score of 0.65 to an electrofishing equipment sample was detected. After all identifications were completed, the system marked 9 suspected target locations and automatically generated an inspection report containing 9 records. Subsequently, law enforcement personnel navigated to the scene one by one based on the coordinates provided in the report to complete the verification and handling.
[0022] This complete inspection operation consists of two phases: flight and identification. Only one operator is needed to monitor the entire process, eliminating the need for multiple ships and personnel. The identification process can run automatically during non-working hours, and the identification results directly generate a report with coordinates, avoiding the problems of missed detections and fatigue caused by manually reviewing the footage frame by frame.
[0023] Example 2: The difference from Example 1 is that, as shown in Example 2... Figure 2As shown, a small reservoir has a total area of 3.5 square kilometers and a total shoreline of 11 kilometers. The main problem with this reservoir is that people frequently set up fixed gillnets in the shallow areas along the shore. This system adopts a simplified configuration scheme, using a consumer-grade quadcopter drone platform equipped with a standard 4K aerial camera, and an onboard GPS module operating in single mode. Both the ground control terminal and the backend image recognition processing terminal are deployed on the same ordinary office laptop, configured with an Intel Core i5 quad-core processor with a clock speed of 2.8GHz and 8GB of RAM. The operator imports an offline map of the reservoir area, marks 42 waypoints along the main water surface and shoreline to generate a closed inspection route, selects the configuration group corresponding to a flight altitude of 100 meters in the flight parameter configuration table, sets the flight speed to 3 meters per second and the shooting interval to 3 seconds, and sets the gimbal camera to tilt downwards at 15 degrees for better observation of the shoreline area. The coordinate watermark position is selected in the upper left corner.
[0024] The fully automated flight inspection took approximately 40 minutes and automatically captured 800 images. All images were continuously transmitted and stored on the laptop's hard drive during the flight. When establishing the sample database, fishery administration personnel collected only 10 photos of each of the three most common types of illegal wire mesh fishing gear, tailored to the specific conditions of the reservoir. After the operator selected and labeled these photos, 30 sample records were generated. Each sample record contained a 12-dimensional feature vector composed of the pixel ratios of the RGB color channels in each interval. After the flight mission, a batch recognition process was initiated. Each image had a resolution of 4000×3000 pixels. Since the target type in this reservoir was singular—only wire mesh fishing gear needed to be identified—the operator set the detection window size to 300×300 pixels with a step size of 150 pixels to increase the detection density. Simultaneously, the sample database containing only 30 records significantly reduced the computational load for a single comparison. Approximately 500 detection windows were extracted from each image based on the window size and step size. After extracting the feature vectors from each window, the similarity score was obtained by calculating the average absolute error with the 30 sample records, while the preset threshold remained at its default value of 0.6. Since laptops have lower performance than industrial PCs and only use four threads for parallel processing, the entire recognition process takes relatively little time. This configuration is particularly suitable for small reservoirs with a single target type and a small inspection range. By reducing the sample library size and optimizing the detection window parameters, the recognition task can be completed on a regular laptop.
[0025] The identification results showed that a similarity of 0.71 to a wire mesh fishing gear sample was detected in a detection window of image 562, 0.64 in image 685, and 0.68 in image 743, totaling three suspected targets. The system marked these three coordinate locations on the electronic map, all pointing to the shallow waters along the west and south banks of the reservoir. Operators reviewed the images one by one to confirm the presence of fixed wire meshes laid in the shallow water and added the three results to the inspection report. The next day, fishery administration personnel used the report to navigate directly to the three marked coordinate locations, confirming on-site that two of them indeed contained illegal wire meshes and removing them. The third location was a false alarm; it was fishing nets drying on the shore, not laid in the water. The overall equipment investment was low. The drones and laptops used commercially available standard products, the software was developed based on an open-source image processing library with no licensing fees, and it did not require a dedicated network line, supporting full offline operation. It is suitable for the budget and technical capability requirements of grassroots fishery administration stations and can be promoted and applied in multiple small reservoirs.
[0026] Example 3: Unlike Example 1, this system is an adaptive dimensionality reduction recognition system. A large reservoir covers an area of 18 square kilometers. The water exhibits significant color variations due to seasonal changes in sediment content from upstream, appearing clear and bluish-green in spring and turbid and yellowish-brown in summer. The fisheries administration department uses this adaptive dimensionality reduction recognition system for routine inspections. The system consists of three parts: a drone terminal, a ground control terminal, and a backend image recognition processing terminal. The drone terminal is equipped with a hexacopter flight platform, a 20-megapixel visible light camera, and a BeiDou positioning module, with a single flight time of 35 minutes. The ground control terminal is a 12-inch ruggedized tablet with built-in flight control software. The operator marks 156 waypoints on an electronic map to generate an inspection route covering the entire reservoir area, sets the flight altitude to 150 meters, the flight speed to 4.5 meters per second, and the shooting interval to 4.5 seconds. The camera maintains a vertical downward shooting position, and the photos are automatically overlaid with a watermark in the lower right corner before being transmitted in real-time to the backend processing terminal for storage via a data transmission link. The backend image recognition processing terminal is deployed in the industrial control computer in the fisheries administration station office, equipped with an Intel Core i7 processor and 32GB of memory.
[0027] The core innovation of the system lies in the adaptive dimensionality reduction mechanism and dynamic feature extraction process of the sample library. During the sample library establishment phase, fishery administration personnel collected 84 photos of the appearance of 7 common types of small fishing boats (12 photos of each type, totaling 84), 30 photos of 3 types of illegal wire mesh nets (10 photos of each type, totaling 30), and 8 photos of electrofishing equipment, for a total of 122 sample photos. The operator opened each sample photo, dragged the mouse to select the rectangular area containing the target object on the screen, entered category labels such as Type A boat, Type B boat, and Type C net, and clicked save. The system then read all pixel data within the rectangle and, following standard procedures, counted the number of pixels in the red channel within the intervals 0-63, 64-127, 128-191, and 192-255. The green and blue channels were counted in the same way. Each of the three channels and four intervals yielded 12 values. Each value was divided by the total number of pixels within the rectangle to obtain a ratio, forming a 12-dimensional feature vector, which, along with the category labels, was written into the sample library file.
[0028] After feature extraction of 122 samples is completed, the system automatically starts the dimensionality reduction analysis program. The program calculates the variance of the numerical distribution of all 122 samples in each dimension, sorts the 12 dimensions in descending order of variance, directly retains the top 7 dimensions with the largest variance as effective dimensions, and removes the bottom 5 dimensions with the smallest variance. The actual analysis results of the spring sample database show that the red channel (128-191) with a variance of 0.0156 is ranked first and retained; the green channel (64-127) with a variance of 0.0198 is ranked second and retained; the green channel (128-191) with a variance of 0.0176 is ranked third and retained; the blue channel (64-127) with a variance of 0.0143 is ranked fourth and retained; the red channel (64-127) with a variance of 0.0089 is ranked fifth and retained; the blue channel (128-191) with a variance of 0.0091 is ranked sixth and retained; and the green channel (192-255) with a variance of 0.0067 is ranked seventh and retained. The remaining five dimensions with variances less than 0.005 were discarded. The system writes the numbers of the seven retained dimensions into the configuration file, and subsequent identification processes only extract and compare the feature values of these seven dimensions.
[0029] On April 15th, the drone conducted its first spring inspection mission. It flew automatically along a preset route for 180 minutes, requiring three battery changes, and captured 3200 images for storage. After the flight, the operator initiated a batch recognition program. The program first read the configuration file to confirm that the currently enabled feature dimensions were 7. It then traversed the 3200 images one by one. Each image had a resolution of 5472×3648 pixels. Detection windows were extracted according to a window size of 400×400 pixels and a step size of 200 pixels: 26 windows were extracted horizontally and 17 vertically, for a total of 442 windows per image. For each detection window, only the feature values of the 7 enabled dimensions were extracted to form a 7-dimensional vector. The 7-dimensional feature vectors of 122 samples in the sample library were read. The similarity between the detection window vector and the sample vector was calculated using the mean absolute error method. The absolute values of the seven corresponding positions of the two 7-dimensional vectors were subtracted, summed, and divided by 7 to obtain the average difference value. The similarity score was obtained by subtracting the average difference value from 1. The system iterates through 122 sample records, recording the highest similarity value and its corresponding category, and stores the highest similarity value and category label for that window. Because the feature dimension was reduced from 12 to 7, the total number of subtraction, addition, and division operations required for a single vector comparison was reduced by 41.7%. However, the overall recognition time is still affected by steps such as file reading, window extraction, and result writing. In actual testing, the recognition process for 3200 images was completed in approximately 5.8 hours on an industrial control computer, which is about 35% shorter than the 8.9 hours taken in the control test using all 12-dimensional features under the same hardware conditions.
[0030] After identification, the system generates a result file containing the similarity scores of all detected windows. The operator can open the visualization interface and adjust the lower limit of the similarity display using a slider. When set to 0.6, the interface only displays the locations of detected windows with a similarity higher than 0.6. Actual inspection revealed that image 1834 detected a suspected target with 0.74 similarity to a Type A vessel sample; image 2156 detected a suspected target with 0.69 similarity to a Type C net sample; and image 2890 detected a suspected target with 0.67 similarity to an electrofishing equipment sample, marking a total of 11 suspected locations. After the operator reviewed and confirmed each location, a PDF inspection report was exported. Law enforcement officers went to the site to verify the coordinates in the report. Of the 11 suspected targets, 9 were confirmed as genuine violations, and 2 were false alarms. The false alarms mainly originated from legal aquaculture management vessels moored on the shore.
[0031] Entering July, the water body became turbid and yellowish-brown due to upstream soil erosion. Because the overall yellowish color of the water rendered the predominantly blue-green characteristic dimensions in the spring sample database ineffective, fisheries officials decided to rebuild the sample database specifically for summer water conditions. Considering that illegal fishing activities are relatively less frequent during the high-temperature summer period and that some vessel types are already sufficiently represented in the spring sample database, the summer sample database was streamlined to include 60 photos of 6 types of fishing vessels (10 photos each), 16 photos of 2 types of gillnets (8 photos each), and 6 photos of electrofishing equipment, totaling 82 summer samples. Operators extracted the 12-dimensional original feature vectors using the same procedure, writing them into the new summer sample database file. The system automatically initiated dimensionality reduction analysis, calculating and sorting the variances of the 82 samples across the 12 dimensions, directly retaining the 6 dimensions with the largest variances as the effective dimensions for the summer sample database.
[0032] The dimensionality reduction analysis results for the summer sample database show that the red channel interval 128-191, with a variance of 0.0201, ranked first and retained; the red channel interval 192-255, with a variance of 0.0187, ranked second and retained; the green channel interval 128-191, with a variance of 0.0165, ranked third and retained; the green channel interval 192-255, with a variance of 0.0143, ranked fourth and retained; the blue channel interval 0-63, with a variance of 0.0112, ranked fifth and retained; and the blue channel interval 64-127, with a variance of 0.0089, ranked sixth and retained. The remaining six dimensions with relatively small variances were removed. The number of enabled dimensions changed to six in summer, and the specific combination of dimensions is significantly different from that in spring. The green channel interval 64-127, retained in spring, did not rank in the top six in summer and was removed, while the blue channel interval 0-63, removed in spring, became the fifth most effective dimension in summer. The system updated the configuration file to record the new enabled dimension numbers.
[0033] On July 20th, during a summer inspection mission, 2800 images were captured. During batch recognition, the system read the configuration file to confirm the activation of 6-dimensional features. Each detection window extracted only the 6-dimensional feature values and compared them with the 6-dimensional vectors of 82 samples in the sample library, further reducing the number of comparison operations per operation. The recognition of 2800 images took approximately 4.1 hours. After setting the display lower limit to 0.6, the operator reviewed the results and marked 8 suspected targets. On-site verification confirmed 6 actual violations and 2 false alarms. The false alarms were floating tree branches in the yellowish-brown water exhibiting a color similar to fishing nets. Comparing data from spring and summer, although the number of dimensions used decreased from 7 to 6 in summer and the combination of dimensions changed significantly, the recognition accuracy remained at a similar level of around 75%, demonstrating that the adaptive dimensionality reduction mechanism can automatically select the most discriminative feature dimensions based on changes in water conditions.
[0034] The key technical point of this solution lies in reversing the redundancy problem deliberately avoided by traditional solutions. Traditional fixed 12-dimensional feature schemes pursue complete feature description, assuming all 12 dimensions are indispensable and forcibly extracting all dimensions for comparison regardless of water conditions. This approach ignores the fact that the contribution of each dimension varies significantly across different scenarios. Some dimensions exhibit highly concentrated, almost constant numerical distributions within specific sample sets. Including these low-variance dimensions in the comparison not only increases the computational burden but also dilutes the weight of high-discrimination dimensions. This solution takes the completely opposite approach, no longer treating the 12-dimensional features as an indivisible whole. Instead, it proactively acknowledges the existence of redundant dimensions with low contribution to the current sample library. It quantifies effectiveness by statistically analyzing the variance of each dimension, automatically eliminating redundant dimensions with small variations, and retaining only dimensions with high variance for comparison. While traditional solutions desperately avoid losing any dimensional information for fear of missing key features, this solution argues that proactively discarding low-value dimensions allows the system to focus on truly effective features. The existence of redundancy is used to reverse-optimize feature extraction, automatically forming the most suitable feature combination under different water conditions.
[0035] Example 4: The difference from Example 1 is that, as shown in Example 4... Figures 3 to 5 As shown, this embodiment proposes an improvement solution to address the problems of heavy computational load in batch recognition and the resource consumption of low-quality images. A passive mechanical tilt recording device is added to the middle of the UAV fuselage. This device consists of a counterweight pendulum, a universal bearing assembly, an angle encoder, and a data recording module.
[0036] The counterweight pendulum uses a cylindrical copper counterweight block 1, with a diameter of 12 mm, a length of 40 mm, and a total weight of 28 grams. The upper end of the counterweight block is connected to the universal bearing assembly via a stainless steel pendulum rod 2 with a diameter of 3 mm. The universal bearing assembly 3 adopts a two-layer nested structure. The outer bearing is fixed to the UAV fuselage support, allowing the internal structure to rotate in the roll direction. The inner bearing is nested inside the outer bearing, allowing the pendulum rod to rotate in the pitch direction. The rotation centers of the two bearings are perpendicular to each other, forming a two-degree-of-freedom universal support. During the inspection flight, the flight is mainly uniform straight-line flight, with a flight speed of about 3.5 meters per second. The turning maneuver period accounts for less than 5%, and the flight altitude remains close to the set value with a variation range of less than 3 meters. The vertical and horizontal acceleration changes are small. Under these small acceleration conditions, the counterweight pendulum maintains an approximately vertical downward direction. A photoelectric angle encoder 4 is installed on the rotation axis of the outer bearing to measure the roll angle, and another photoelectric angle encoder 4 is installed on the rotation axis of the inner bearing to measure the pitch angle. The resolution of both encoders is 0.1 degrees. In actual inspection flights, the range of fuselage tilt angle changes is usually between 0 and 12 degrees, and the instantaneous tilt angle may reach about 18 degrees when encountering airflow disturbances.
[0037] The data logging module is connected to the shutter trigger signal of the airborne camera. Each time the camera takes an exposure, the data logging module synchronously reads the angle output values from the pitch and roll encoders, calculates the square root of the sum of the squares of the two angles to obtain a composite tilt angle assessment value, reflecting the degree to which the aircraft attitude deviates from a horizontal state. The data logging module writes the image file name and the corresponding composite tilt angle assessment value into an attitude log file in chronological order of shooting time. The file is in text format, with each line containing three fields separated by commas: timestamp, image file name, and composite tilt angle assessment value. During a 120-minute inspection mission, 1800 images were captured. The generated attitude log file contained 1800 lines and was approximately 80KB in size, stored on the airborne memory card and transferred to the backend processing unit along with the image files. The entire system is powered by the airborne power supply, and the total power consumption, including the two angle encoders and the data logging module, is less than 1.2 watts, with negligible impact on flight endurance. This device operates independently and is not connected to the inertial measurement unit (IMU) of the flight control system. The IMU data of the flight control system is usually used for flight attitude stabilization control and the data interface is not open to the outside. Adding an independent mechanical tilt recording device can avoid modifying the flight control software and data bus structure, making it suitable for rapid installation on existing platforms.
[0038] Before batch recognition, the backend image recognition processing unit adds an image quality pre-screening module. After reading the attitude log file, it sorts the images by composite tilt angle evaluation values from smallest to largest. A small composite tilt angle evaluation value indicates that the aircraft attitude was stable during shooting, with minimal impact from camera vibration and motion blur, resulting in high image sharpness. A large composite tilt angle evaluation value indicates that the aircraft attitude changed drastically during shooting, potentially leading to blurry images or misaligned compositions. Based on manual review of image sharpness corresponding to different tilt angle values during multiple actual flight tests, the system sets a quality grading standard: composite tilt angle evaluation values less than 3 degrees are defined as high-quality images, 3 to 8 degrees as medium-quality images, and greater than 8 degrees as low-quality images.
[0039] In a certain task, out of 1800 images, 1020 were high-quality (57%), 580 were medium-quality (32%), and 200 were low-quality (11%). The batch recognition process prioritized high-quality images, passing a list of filenames for the 1020 high-quality images to the recognition program. Each image had a resolution of 5472×3648 pixels, a detection window size of 350×350 pixels with a step size of 175 pixels, and approximately 620 detection windows were extracted per image, totaling approximately 630,000 detection windows across the 1020 images. The backend processing unit used an eight-core processor with multi-threaded parallel processing, completing the recognition of the 1020 high-quality images in approximately 2.8 hours and generating a preliminary inspection report. If the operator found abnormal distribution of suspected targets or omissions in certain areas after reviewing the report, a secondary recognition process could be initiated to process the 580 medium-quality images, taking approximately 1.6 hours. The results were then merged into the complete inspection report. Low-quality images were not included in the recognition process by default, but their filenames and coordinate information were retained for manual spot checks. The pre-screening process took approximately 7.2 hours to process all 1,800 images, while the process of prioritizing high-quality images after pre-screening was shortened to 2.8 hours.
[0040] In a practical application, a mission encountered a Force 4 gust of wind. At the 68-minute mark, the attitude log showed that the composite tilt angle assessment value began to fluctuate. The entire process, from the onset of fluctuation to stabilization, lasted approximately 120 seconds. The peak fluctuation phase lasted about 28 seconds, with the tilt angle reaching a peak of 14 degrees. Before and after the peak, there were approximately 46-second attitude transition and recovery phases, with the tilt angle fluctuating between 3 and 8 degrees. During the entire 120-second period, approximately 34 images were captured at 3.5-second intervals. Eight images from the peak phase were marked as low quality, and 26 images from the transition phase were marked as medium quality. These 34 images, at a flight speed of 3.5 meters per second, corresponded to a ground coverage length of approximately 420 meters. The operator arranged for a re-flight and re-capture of the area the following day to ensure complete coverage.
[0041] Image quality pre-screening is achieved through a mechanical tilt recording device, requiring only the processing of approximately 60% of high-quality images in daily tasks to cover the effective inspection area. The counterweight pendulum device weighs approximately 35 grams, and the angle encoder uses a standard industrial sensor with high reliability. This solution differs from traditional approaches. Traditional solutions, facing heavy computational loads, typically upgrade processor hardware or optimize recognition algorithms to increase the ability to process all images. This solution, however, reverses this constraint by adding a simple mechanical sensing device to complete quality labeling and grading during image acquisition. This transforms the original attempt to process all images into a strategy of prioritizing high-quality images, replacing complex image quality assessment algorithms with the passive sensing capabilities of the mechanical device, thus reducing computational load while ensuring effective coverage.
[0042] Example 5 differs from Example 1 in that, as Figures 6 to 8As shown, this embodiment addresses the issues of water surface reflection interference and insufficient shoreline coverage by adding fixed-angle auxiliary cameras 7 to both sides of the main gimbal camera 6 on the underside of the UAV 5. The left auxiliary camera 7 is mounted 18 centimeters away from the main gimbal camera 6 via an aluminum alloy mounting bracket 8, with its lens optical axis tilted 25 degrees to the left relative to the vertical direction. The right auxiliary camera 7 is symmetrically mounted and tilted 25 degrees to the right. The left and right auxiliary cameras are miniature action cameras with 1 / 2.3-inch sensors, an image resolution of 4000×3000 pixels, and each camera, including the bracket 8, weighs approximately 80 grams. The three cameras are synchronously exposed via a unified trigger signal from the flight controller. The main camera image file name is IMG_1247.JPG, the left auxiliary camera image is automatically appended with the suffix "L" to become IMG_1247_L.JPG, and the right auxiliary camera image is appended with the suffix "R" to become IMG_1247_R.JPG.
[0043] At a flight altitude of 120 meters, the main camera captures a vertical image covering an area of approximately 220 meters by 220 meters on the ground. The auxiliary camera on the left is tilted at a 25-degree angle, shifting its image center approximately 56 meters to the left relative to the main camera. Due to perspective projection, the ground coverage area appears as an approximately elliptical shape stretched in the tilted direction. The auxiliary camera on the right is symmetrically shifted to the right, covering the area on the right side. This three-camera spatial distribution allows a single flight to simultaneously cover the central water surface and the shallow shorelines on both sides, eliminating the need for additional shoreline-specific flight paths.
[0044] The backend processing module adds a multi-view differential preprocessing module, which reads three image files from the same shooting moment and calculates the overlap area on the ground between the left camera image and the main camera image based on a flight altitude of 120 meters, camera spacing of 18 centimeters, and tilt angle of 25 degrees. The camera spacing and tilt angle are fixed installation parameters, and the perspective transformation matrix only changes with flight altitude. The system pre-calculates the perspective transformation matrix parameters corresponding to flight altitudes between 100 and 150 meters at 2-meter intervals and stores them as a lookup table with 26 entries. Pixel blocks of the overlapping area in the left image and the corresponding pixel blocks in the main camera image are extracted. Since the auxiliary camera's resolution differs from the main camera's, the overlapping blocks in the left image are first resampled to the same pixel size as the main camera image blocks using bilinear interpolation. Then, based on the flight altitude recorded in the image, the corresponding perspective transformation matrix is selected after rounding to the nearest table entry height value for geometric correction to compensate for perspective distortion caused by tilted shooting. After correction, the two pixel matrices correspond to the same ground position and can be compared pixel by pixel.
[0045] The left-side difference value is obtained by calculating the absolute value of the brightness difference between the main camera image and the left-side image pixel by pixel. The difference value ranges from 0 to 255, corresponding to the maximum difference in pixel brightness. Water surface reflection has specular reflection characteristics, and the intensity of reflected light changes significantly with the viewing angle. When shooting vertically and at an angle, the intensity of reflected light received in the same reflective area differs due to the difference in viewing angle, which is manifested as a brightness difference in the image, resulting in a large difference value. The surface of real fishing boats or fishing gear is mostly made of diffuse reflective material, and diffuse reflective light is scattered uniformly in all directions. The brightness change is small when shooting from different angles, resulting in a low difference value. The system sets a discrimination threshold of 60. Pixels with a left-side difference value exceeding 60 are marked as suspected reflections. The right-side auxiliary camera image and the main camera image are subjected to the same difference calculation to obtain the right-side difference value. Pixels with a right-side difference value exceeding 60 are also marked as suspected reflections. For the same pixel location, if either the left or right difference value exceeds 60, the pixel is marked as a reflective area. All pixel locations marked as reflective are aggregated to generate a difference mask map.
[0046] In subsequent batch recognition processes, when traversing the detection windows of the main camera images, the differential mask image is first queried. If more than 40% of the pixels in a detection window are marked as reflective areas, that window is skipped without feature extraction or comparison with the sample library. During a patrol at 10:00 AM, a bright reflection occupying approximately 15% of the water surface in the image of main camera number 1035 appeared. The average difference value calculated from the left auxiliary image was 112, exceeding the threshold of 60. The average difference value calculated from the right auxiliary image was 108, also exceeding the threshold of 60. This area was confirmed as reflective, and the 78 detection windows within the area were automatically skipped to avoid false detections. A small fishing boat's hull, captured at the edge of the same image, has a diffuse reflective surface. The difference value with the left auxiliary image was 19, and the difference value with the right auxiliary image was 24. Both the left and right difference values were below the threshold of 60, and the boat was not marked as reflective. The recognition process proceeded normally, and the target was successfully detected. The images of the shoreline and shallow water areas captured by the left and right auxiliary cameras are also processed for target recognition separately. The recognition is carried out according to the same detection window division and feature extraction process as the main camera. The recognition results are merged with the recognition results of the main camera into the same inspection report.
[0047] Actual measurements show that the false detection rate decreased from 18% to 5% after enabling differential preprocessing under strong light conditions at noon. The number of fixed fishing nets detected by the left and right auxiliary cameras on the shoreline and shallows increased by about 30% compared to the single vertical shooting mode. The total weight of the two auxiliary camera modules and the mounting bracket is about 170 grams, which has little impact on the payload of the multi-rotor UAV. The simultaneous shooting of the three cameras tripled the number of image files. After differential preprocessing to remove reflective areas, the number of effective detection windows in the main camera image decreased by about 12%, reducing the recognition computation load of the main camera image. However, the images from the left and right auxiliary cameras need to participate in differential calculation and shoreline area recognition. Overall, the computation load of the three cameras increased, and the actual processing time increased by about 40% compared to the single-camera mode, but the recognition accuracy was significantly improved and the coverage area was significantly expanded. This solution differs from traditional approaches. Traditional solutions, when faced with the problem of a single gimbal angle, typically increase the gimbal's degrees of freedom to achieve multi-angle adjustable shooting or configure multiple independent gimbals for separate control. This solution, however, reverses this seemingly limiting feature of a fixed gimbal angle. By adding two auxiliary cameras with fixed tilt angles to a fixed main gimbal, a stable multi-view geometric relationship is established. By using the differences in pixel brightness of the water surface reflection area under different viewpoints, differential calculations are performed to identify and eliminate reflection interference. The original requirement of adjusting the gimbal angle to avoid reflection is transformed into the ability to actively identify reflection through fixed-angle multi-views. At the same time, the fixed angle relationship allows geometric correction parameters such as the perspective transformation matrix to be pre-calculated based on the flight altitude and stored as a lookup table, reducing the real-time calculation burden.
[0048] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A drone-based intelligent inspection and image recognition system for fisheries patrols in large reservoirs, characterized in that: It consists of a drone terminal, a ground control terminal, and a back-end image recognition processing terminal. The ground control terminal consists of a drone flight controller and a touch screen. The ground control terminal is connected to the drone terminal via signal. The back-end image recognition processing terminal receives photos taken by the drone terminal, processes them, and generates an inspection report.
2. The UAV intelligent inspection and image recognition system for fisheries patrol of large-scale reservoirs according to claim 1, characterized in that, The UAV is equipped with a multi-rotor flight platform with at least one visible light gimbal camera and an onboard positioning module. The back-end image recognition processing terminal is deployed in a local computer and includes a fishery patrol target image sample library and a task data storage module. During the flight of the UAV along a preset route, the gimbal camera automatically takes photos at preset time intervals, overlays coordinate watermarks, and transmits them to the back-end storage. The back-end image recognition processing terminal divides the image into multiple detection windows, extracts color feature vectors from each window, and calculates the similarity with the feature vectors of each sample in the sample library. Detection windows with similarity exceeding a preset threshold are identified as suspected target areas and their coordinate information is recorded. After recognition, an inspection report containing the location coordinates of the suspected target and the image file is generated.
3. The intelligent drone inspection and image recognition system for fisheries patrol of large-scale reservoirs according to claim 2, characterized in that, The method for extracting the color feature vector is to count the number of pixels in each of the three RGB color channels in the image area within four intervals: 0 to 63, 64 to 127, 128 to 191, and 192 to 255. The number of pixels in each interval is divided by the total number of pixels in the area to obtain 12 ratio values to form the feature vector.
4. The UAV intelligent inspection and image recognition system for fisheries patrol of large-scale reservoirs according to claim 3, characterized in that, The similarity calculation method is to subtract the 12 values at the corresponding positions of the detection window feature vector and the sample feature vector respectively, take the absolute value and then sum them. Divide the sum by 12 to get the average difference value, and subtract the average difference value from 1 to get the similarity score.
5. The intelligent drone inspection and image recognition system for fisheries patrol of large-scale reservoirs according to claim 4, characterized in that, The detection window is square, with equal width and height, and is set between 200 and 500 pixels. It is extracted from left to right and from top to bottom with a fixed step size, which is set to half the side length of the detection window so that adjacent windows overlap by half.
6. The UAV intelligent inspection and image recognition system for fisheries patrol of large-scale reservoirs according to claim 5, characterized in that, The drone's flight parameters include flight altitude, flight speed, and shooting interval. The system's built-in flight parameter configuration table lists recommended flight speeds and shooting intervals for different flight altitudes for the operator to select.
7. The intelligent drone inspection and image recognition system for fisheries patrol of large-scale reservoirs according to claim 6, characterized in that, The preset threshold can be adjusted by the operator in the system configuration interface. The default value is set to 0.6, and the value range is between 0.4 and 0.
9. The sample library index file uses text format to record the information of one sample per line, including the category label and 12 feature values. Multiple samples are appended to form a complete sample library.
8. The intelligent drone inspection and image recognition system for fisheries patrol of large-scale reservoirs according to claim 7, characterized in that, The coordinate watermark position can be selected as the top left, top right, bottom left, or bottom right corner in the system configuration interface; the back-end image recognition processing terminal adopts a multi-threaded parallel processing method to process multiple images simultaneously to improve recognition efficiency; the inspection report is exported in PDF format, which includes the file name, shooting coordinates, matched sample category label, and similarity score value of each suspected target area image.
Citation Information
Patent Citations
A method and system for monitoring prohibited fishing gear based on drones
CN118279773B
Camera shooting unmanned aerial vehicle for reservoir quality inspection and use method of camera shooting unmanned aerial vehicle
CN118894250A
Reservoir safety intelligent inspection method and system based on unmanned aerial vehicle
CN119295983A
