A collaborative method for monitoring and managing golden apple snail egg masses using low-altitude unmanned aerial vehicles (UAVs) and unmanned surface vessels (USVs).

Abstract

This invention provides a method for monitoring and managing golden apple snail egg masses using a low-altitude unmanned aerial vehicle (UAV) and an unmanned surface vessel (USV). The method involves using the UAV to acquire images and locate coordinates of golden apple snail egg masses in the monitored water area to construct a preliminary distribution map. Then, the accurate location coordinates are divided using the DBSCAN clustering method, thereby dividing the monitored water area into zones with different ecological risk levels. Based on the ecological risk level zone division information, a density hotspot priority list for golden apple snail egg mass removal tasks is established, prioritizing high-risk, urgent areas with high egg mass density. Combined with weather, water flow, and obstacle factors in the monitored water area, the USV plans the removal route for the golden apple snail egg masses. The USV then accurately removes the golden apple snail egg masses in the monitored water area according to the removal route. This method can quickly identify and remove golden apple snail egg masses in a wide range of water areas, thereby achieving dynamic monitoring and continuous management, effectively protecting the aquatic ecological environment and improving water quality recovery capabilities.

Description

Technical Field

[0001] This invention belongs to the field of data processing systems or methods for monitoring or prediction purposes, and specifically relates to a method for monitoring and managing golden apple snail egg masses in collaboration with low-altitude unmanned aerial vehicles and unmanned surface vessels, as well as a method for preparing such masses. Background Technology

[0002] With the advancement of urbanization, many urban rivers have suffered severe pollution, and the aquatic ecological environment is deteriorating. As an invasive species, the golden apple snail's egg masses proliferate in large numbers in waterways, not only damaging water quality but also affecting the growth of aquatic species and even endangering human health. Traditional removal methods rely on manual and mechanized operations, which suffer from low efficiency, narrow coverage, and high costs, making them unsuitable for addressing the needs of large-scale waterway management.

[0003] In conclusion, a method is urgently needed to develop that can quickly identify and remove golden apple snail egg masses in a wide range of water areas, thereby enabling dynamic monitoring and continuous management to effectively protect the aquatic ecological environment and improve water quality recovery capabilities. Summary of the Invention

[0004] In view of the shortcomings of the existing technology mentioned above, this paper aims to solve the technical problem that the existing technology cannot quickly identify and remove golden apple snail egg masses to achieve dynamic monitoring and continuous management, so as to effectively protect the aquatic ecological environment and improve the water quality recovery capacity.

[0005] The present invention also aims to provide a method for monitoring and controlling golden apple snail egg masses in collaboration with low-altitude unmanned aerial vehicles and unmanned surface vessels, comprising the following steps:

[0006] S1: Use drones to collect images and locate coordinates of golden apple snail egg masses in the monitored water area to construct a preliminary distribution map of golden apple snail egg masses;

[0007] S2: The accurate positioning coordinates are then divided using the DBSCAN clustering method, thereby dividing the monitored water area into zones with different ecological risk levels;

[0008] S3: Based on the ecological risk level area division information, set up a density hotspot priority list for the Golden Apple Snail egg mass removal task, prioritize the handling of high ecological risk emergency areas with high Golden Apple Snail egg mass density, and combine the weather, water flow, and obstacle factors in the monitored water area to complete the unmanned vessel route planning for Golden Apple Snail egg mass removal.

[0009] S4: The unmanned vessel accurately and thoroughly removes the golden apple snail egg masses in the monitored waters according to the removal route;

[0010] S5: After the unmanned surface vessel (USV) completes its initial cleanup of the monitored waters, it returns to its starting point to await further instructions. The USV then re-flys at high altitude to collect images of the monitored waters and uploads the image data to the cloud in real time for egg mass identification and cleanup effectiveness assessment. The cloud compares the re-fly data with previous cleanup records to determine if the egg masses have been completely removed and updates the ecological risk zoning map of the waters based on the results. If it finds that egg masses have not been removed in certain areas, the cloud will adjust the operational path and priority, and reassign the USV to perform the cleanup task again until all cleanup tasks are completed.

[0011] Image acquisition in S1 includes the UAV first flying at low altitude along the monitoring area, completing continuous environmental mapping and acquiring images of golden apple snail egg masses using SLAM, and then ascending to high altitude to acquire global images of the monitoring area; wherein...

[0012] During the acquisition of images of golden apple snail egg masses, YOLOv5 image recognition was used in the cloud to detect, identify, and remove impurities from the egg masses, resulting in a monitoring and positioning coordinate map of the egg masses; the NWD loss function was added to the YOLOv5 image recognition; and...

[0013] The risk level classification is as follows: After the DBSCAN clustering method is completed, the density values ​​of golden apple snail egg masses are statistically analyzed for several clusters obtained by clustering. The clusters are sorted according to the density values ​​of golden apple snail egg masses. The 80% quantile is selected as the critical value for high ecological risk areas; the 40% quantile is selected as the critical value for medium ecological risk areas; and areas with less than 40% quantile are considered low ecological risk areas.

[0014] Preferably, the low-altitude range is 8 to 20 meters; the high-altitude range is 30 to 80 meters.

[0015] Preferably, in step S4, when the unmanned surface vessel (USV) arrives at the designated removal area, it first needs to perform a secondary location confirmation of the golden apple snail egg masses in that area. If the USV egg masses are found to be consistent with the corresponding positions on the monitoring and positioning coordinate map, the USV egg masses are removed by high-speed, high-pressure water jets from the water guns mounted on the USV. If inconsistencies are found, the position of the USV needs to be fine-tuned until the corresponding USV egg masses are identified, and then removal is performed. If the corresponding USV egg masses still cannot be identified after multiple fine-tunings, the USV moves to the next removal area and repeats the above steps until the removal task is completed.

[0016] Preferably, the removal of golden apple snail egg masses includes information acquisition, control decision-making, and execution adjustment; wherein, information acquisition includes acquiring the positioning information of the target golden apple snail egg masses and the water gun spray angle information; control decision-making includes the angular error and distance error between the water gun pointing and the target, and the PID controller in the information processing module processes the two types of errors and generates control signals; execution adjustment includes driving the motor to adjust the horizontal and vertical angles of the gimbal according to the control signals, dynamically adjusting feedback, and controlling the PWM water pump spray.

[0017] Preferably, the location information of the target golden apple snail egg mass and the water jet angle information are obtained:

[0018] With the camera as the origin, the three-dimensional spatial coordinates of the target golden apple snail egg mass relative to the camera are P. c =(x c y c , z c );

[0019] With the water gun as the origin, the three-dimensional spatial coordinates of the target golden apple snail egg mass relative to the water gun are P. w =(x w y w , z w );

[0020] camera intrinsic parameter matrix Among them, f x and f y y is the focal length of the camera, and cx and cy are the coordinates of the camera's optical center.

[0021] When a mass of golden apple snail eggs appears in the field of view of a depth camera, assuming the camera detects the two-dimensional pixel coordinates of the target as (u, v), then the depth information is obtained using the intrinsic parameter matrix k and depth information z of the depth camera. c This allows us to obtain the three-dimensional spatial coordinates of the target object in the camera coordinate system:

[0022]

[0023] Since the relative spatial positions of the camera and the water gun remain unchanged, the camera's extrinsic parameter matrix T can be used to determine this. cw Transform the target object from the camera coordinate system to the water gun coordinate system:

[0024] P w =T cw ·P c ;in,

[0025] in It is a 4x4 rigid body transformation matrix; including the rotation matrix R. 3x3 Translation vector t 3x1 ;

[0026] Based on the three-dimensional spatial coordinates (x) of the target object in the water gun coordinate system w y w , z w ), that is, P w It can calculate the pitch angle θ that the gimbal needs to be adjusted. That is, the rotation angle and horizontal angle that the water gun needs to be adjusted in the vertical plane. This refers to the angle of rotation that the water gun needs to be adjusted in the horizontal plane.

[0027] Preferably, a golden apple snail egg mass growth cycle prediction mechanism is superimposed on the density hotspot priority list.

[0028] Preferably, the golden apple snail egg mass growth cycle prediction mechanism uses an LSTM model to capture the egg-laying pattern of the golden apple snail by analyzing historical environmental climate data, historical water quality data, and historical egg-laying time intervals within the monitoring area, and then predicts the time of the next egg-laying.

[0029] Preferably, the growth cycle prediction mechanism of the golden apple snail egg mass is expressed by the following formula:

[0030] T next =T base +ΔT season +ΔT environment ;in,

[0031] T next It is the predicted time of the next spawning;

[0032] T base This is a baseline value, assuming no other factors affect the spawning time;

[0033] ΔT season The error is caused by seasonal factors;

[0034] ΔT environment The error is caused by water quality and environmental factors;

[0035] For the baseline value, we use the mean model for calculation:

[0036] in, N represents the spawning time observed in historical data; N is the total amount of historical data.

[0037] Preferably, it also includes a safety margin T safetymargin :

[0038] The training error of the LSTM model is measured using MSE, and a weighting factor α is defined. safety To control the size of the safety margin;

[0039]

[0040] in:

[0041] MSE: Mean squared error of the current model prediction;

[0042] MSE max The largest mean square error in history, used as the basis for standardization;

[0043] ∈: A small constant to prevent the denominator from being zero.

[0044] To better consider the baseline value T base The role of introducing a weighting factor β base To compare the difference between the baseline value and the predicted value:

[0045] in,

[0046] T next The next spawning time predicted by the model;

[0047] T base : Baseline value, assuming no other factors affect the spawning time;

[0048] ∈: a small constant to prevent the denominator from being zero;

[0049] β base-max : For β base The maximum value is set to prevent excessive influence on the inspection interval when the prediction error is too large; when the prediction is relatively accurate, it is set to 0.2 to 0.5; when the prediction is unstable, it is set to 0.5 to 1.0.

[0050] Combining α safety With β base Safety margin T safetymargin The final formula is designed as follows:

[0051] T safety margin =T next ×(1-α safety ×MSE)+β base ×|T base -T next |

[0052] Preferably, this also includes the design of the inspection time interval:

[0053] If the predicted value is T next The inspection interval should be:

[0054] T inspection =min(T) nextT base T safety margin );

[0055] in:

[0056] T next The next spawning time predicted by the model;

[0057] T base : Baseline value, assuming no other factors affect the spawning time;

[0058] T safety margin: a safety margin for the prediction time interval based on the uncertainty of the prediction.

[0059] The preparation method of the method for monitoring and controlling golden apple snail egg masses in collaboration with low-altitude unmanned aerial vehicles and unmanned surface vessels presented in this case has the following beneficial effects:

[0060] 1) This invention replaces the traditional IoU loss with the NWD loss function. NWD can effectively solve the problem of inaccurate IoU in the labeling of small target objects, thereby improving the detection accuracy of small targets such as golden apple snail eggs;

[0061] 2) The cloud-based system of this invention has already predicted the distribution of egg masses based on images and coordinates collected by low-altitude UAVs. However, since the distribution of egg masses may undergo certain dynamic changes (such as the influence of water flow, object occlusion, etc.), each egg mass must be reconfirmed. The UAV will use its own vision system to re-identify the specific location, size, and shape of the egg masses to ensure that the removal task is carried out accurately.

[0062] 3) The design of the patrol interval, by combining the predicted spawning interval, the baseline spawning time and the safety margin, provides patrol personnel with a scientific and flexible time arrangement. This approach not only ensures the timeliness of the patrol, but also avoids too many or too few patrols, ensuring the rational use of resources, and ultimately achieving effective monitoring and management of the spawning behavior of golden apple snails.

[0063] 4) Layered and coordinated low-altitude and high-altitude patrols, balancing detailed data collection with a broad overview; In step S1, this invention introduces a monitoring method combining low-altitude and high-altitude flight, achieving both detailed clarity and area coverage: low-altitude images provide the high-resolution images needed for accurate identification, while high-altitude aerial photography allows managers to grasp the overall distribution and hydrological environment. It also effectively reduces missed detections and fills blind spots: during low-altitude patrols, if obstacles obstruct the view, subsequent high-altitude aerial photography can promptly supplement the information, and vice versa; improving patrol efficiency: through the mutual verification of the two types of flight data, the cloud can quickly focus on suspected high-risk areas or missed detection areas, thereby reducing invalid flight paths and shortening the patrol cycle.

[0064] 5) This invention proposes the concept of "cooperative operation of low-altitude UAVs and unmanned surface vessels," which not only refers to the collaborative work of UAVs (airborne vehicles) and unmanned vessels (sea / waterborne vehicles), but also implies that it can be extended to various types of aquatic environments such as coastal areas, estuaries, lakes, and inland rivers. The benefits of this air-water-surface integrated management include: adaptability to diverse aquatic terrain: UAVs and unmanned vessels can flexibly switch operating routes in situations such as shallow waters, winding river channels, and tidal changes; reduced risks of manual operations: no need for a large number of people to go into the water or take a boat to search for egg masses at close range, which is both safe and efficient; rapid response to emergencies: if an extremely high-risk area is found during patrol, the UAV can immediately mark the location, and the unmanned vessel can quickly go to handle it.

[0065] 6) It can be integrated with multi-source environmental monitoring data to form comprehensive water management decision support; In the system of this invention, in addition to the images and coordinate data collected by the UAV and unmanned vessel themselves, multi-source environmental monitoring data (such as air temperature, water temperature, pH, nutrient concentration, river water quality sensor data, etc.) can also be included in the cloud database; More comprehensive risk assessment can be obtained: The cloud can assess the density of golden apple snail egg masses while combining water quality conditions or algae concentration and other indicators to judge the overall health of the water ecosystem; Accurately identify the causes of high egg mass occurrence: If some water areas are caused by high water temperature or eutrophication, resulting in concentrated egg laying by golden apple snails, the management strategy can be changed to make it more targeted, such as subsequent release of biological inhibitors; Long-term monitoring and trend prediction: The accumulated multi-dimensional data can be further trained to train a more accurate time series model, and conduct a more in-depth analysis and prediction of the evolution of water ecosystems and the reproductive patterns of golden apple snails.

[0066] 7) In the traditional manual inspection and retrieval mode, operators need to spend a lot of time and energy to search, record, and remove golden apple snail egg masses. The water-air collaborative system of this invention has a high degree of automation and intelligence; it can save manpower significantly: managers only need to supervise, adjust strategies and perform a small amount of maintenance; it can operate 24 hours a day: compared with manual labor, the intelligent unmanned platform can work continuously at night or in multiple shifts when necessary, shortening the disposal time of golden apple snail egg masses before hatching; and it provides sufficient human safety protection: effectively reducing the risks of dangers encountered during water operations, such as underwater currents and silt sinking. Attached Figure Description

[0067] Figure 1 This is a flowchart illustrating the process of using an unmanned aerial vehicle (UAV) to acquire images, locate, and construct maps of golden apple snail egg masses in a river channel, as described in this invention.

[0068] Figure 2 This is a schematic diagram of the risk zoning of golden apple snail egg masses in urban inland waterways generated by DBSCAN according to the present invention;

[0069] Figure 3This is a flowchart of the egg-laying and patrol mechanism of the golden apple snail based on LSTM (temporal recurrent neural network) according to the present invention;

[0070] Figure 4 Schematic diagram illustrating the working principle of drones and unmanned vessels working together;

[0071] Figure 5 This is a flowchart illustrating the process of unmanned surface vessel (USV) patrolling and removing golden apple snail egg masses according to the present invention.

[0072] Figure 6 This is a schematic diagram illustrating the working principle of the UAV of the present invention;

[0073] Figure 7 This is a schematic diagram illustrating the working principle of the unmanned surface vessel of the present invention.

[0074] Figure 8 This diagram illustrates the working principle of cloud-based collaboration with drones and unmanned vessels. Detailed Implementation

[0075] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification.

[0076] Example 1:

[0077] A method for monitoring and removing golden apple snail egg masses using a combined sea and air approach includes the following steps:

[0078] I. Unmanned Aerial Vehicle (UAV) Monitoring: (e.g., Figure 1 (As shown)

[0079] Visual SLAM Mapping: During drone flight, visual SLAM (Simultaneous Localization and Mapping) technology is used to achieve high-precision self-localization and environmental mapping. The SLAM system uses continuous images captured by a camera and acceleration and angular velocity data provided by an IMU (Inertial Measurement Unit) to calculate the drone's position and attitude in three-dimensional space in real time, thereby constructing a map of the river channel and its surrounding environment. The drone continuously updates the map through movement, while simultaneously matching and optimizing environmental features to ensure the map's accuracy and real-time performance. In this process, visual SLAM enables drones to achieve autonomous localization in urban environments with no or unstable GPS signals.

[0080] Image Acquisition and Coordinate Recording: After completing SLAM mapping, the drone will continue to fly at low altitude along the river, focusing on image acquisition of golden apple snail egg masses on the riverbank. The drone will automatically control its flight path to ensure full coverage of areas where egg masses may exist. While acquiring images, the drone will record the precise coordinate information corresponding to each image. The images and their coordinate data will be uploaded to the cloud in real time to facilitate subsequent image analysis, egg mass identification, and risk assessment.

[0081] Global Image Acquisition: After completing low-altitude flight and egg mass identification, the drone will ascend to a higher altitude to acquire global images, providing an overview of the entire river channel. High-altitude image acquisition helps obtain more comprehensive environmental data, including the overall layout of the river channel, water area, flow velocity, bridges, and other factors that may affect path planning and unmanned vessel operations. The drone will upload the panoramic images acquired from the high altitude to the cloud. The cloud will then use image analysis, along with the number and coordinates of the acquired egg masses, to generate a risk zoning map of the entire river channel using the DBSCAN density clustering algorithm.

[0082] YOLOv5 Image Recognition: Image data collected by the drone is uploaded to a cloud server in real time. The cloud server uses the YOLOv5 image recognition model to detect and identify golden apple snail egg masses in the images. Since golden apple snail egg masses are small targets, the NWD (Normalized Wasserstein Distance) loss function is introduced on top of the original YOLOv5 to improve detection performance. Unlike the traditional IoU loss function, the NWD loss function is insensitive to objects of different scales, making it more suitable for measuring the similarity between small objects. By replacing the traditional IoU loss, NWD effectively solves the inaccuracy of IoU in label assignment for small targets, thereby improving the detection accuracy of small targets such as golden apple snail egg masses.

[0083] Risk Zoning Generation (DBSCAN Clustering): Based on the coordinates of egg masses identified by YOLOv5, the cloud uses the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm to spatially cluster the egg masses, identifying their distribution patterns in the water area. Through DBSCAN, the cloud can divide the water area into zones with different risk levels, such as... Figure 2 As shown.

[0084] The risk level classification is as follows: After the DBSCAN clustering method is completed, the density values ​​of golden apple snail egg masses are statistically analyzed for several clusters obtained by clustering. The clusters are sorted according to the density values ​​of golden apple snail egg masses. The 80% quantile is selected as the critical value of high-risk area; the 40% quantile is selected as the critical value of medium-risk area; and areas with less than 40% quantile are considered low-risk areas.

[0085] Path planning and task allocation: Based on global image data, precise coordinates of golden apple snail egg masses, and risk zoning results, the cloud-based system will use the A* (A-star) path planning algorithm to generate the optimal route from the unmanned vessel's current location to high-ecological-risk areas. Path planning will consider various factors in the water, such as obstacles, currents, and bridges, to ensure the unmanned vessel can safely and efficiently complete the egg mass removal task. The cloud-based system will also adjust task priorities based on real-time conditions to ensure that the most urgent areas are addressed promptly.

[0086] II. Unmanned surface vessel (USV) patrols to remove apple snail egg masses (e.g.) Figure 5 As shown):

[0087] 1. High-precision positioning and path execution: Unmanned surface vessels (USVs) need to navigate accurately in complex aquatic environments to ensure they can reach the designated target area for egg mass removal operations. To this end, USVs use RTK-GPS (Real-time Dynamic Differential GPS) combined with LiDAR (Light Detection and Ranging) technology for high-precision positioning.

[0088] 2. Task Execution: Based on the task instructions and path planning issued from the cloud, the unmanned surface vessel (USV) will automatically execute the assigned tasks. The cloud generates path planning instructions based on the distribution of egg masses in the water area and the ecological risk level, prioritizing the USV's handling of egg masses in high-risk areas. The USV will travel along the planned path, collecting real-time data on the surrounding environment, avoiding obstacles, and accurately reaching the target location.

[0089] 3. Egg mass removal and high-pressure water gun control

[0090] Secondary Egg Mass Recognition: Once the unmanned surface vessel (USV) reaches the designated work area, it uses its onboard camera to perform secondary egg mass recognition. The cloud-based system has already predicted the egg mass distribution based on images and coordinates collected by the low-altitude drone. However, because the distribution of egg masses may dynamically change (e.g., due to water flow or object obstruction), each egg mass must be reconfirmed. The USV uses its vision system to further identify the specific location, size, and shape of the egg masses to ensure accurate removal. The camera, in conjunction with the YOLOv5 algorithm, performs rapid identification and positioning through real-time image capture, precisely locking the coordinates of the egg masses.

[0091] Egg mass removal: The egg mass removal technology in this design is specifically achieved by a depth-of-field camera to capture and locate the target object, the golden apple snail egg mass, a two-dimensional servo motor to aim at it, and a PWM water pump to spray water at a variable frequency. The specific working principle of this step is divided into four main stages: information acquisition, control decision, execution adjustment, and UAV re-flight and secondary monitoring.

[0092] A. Information Acquisition: The IMU sensor measures the gimbal's attitude angles (e.g., pitch and horizontal angles) in real time. By obtaining the gimbal's current orientation, the control system can determine whether the gimbal's actual attitude matches the expected attitude. The specific process is as follows:

[0093] Target location

[0094] Define the camera coordinate system: with the camera as the origin, and the three-dimensional spatial coordinates of the target object as P. c =(x c y c , z c );

[0095] Define the water gun coordinate system: with the water gun as the origin, the three-dimensional spatial coordinates of the target object are P. w =(x w y w , z w );

[0096] camera intrinsic parameter matrix f x and f y denoted as the focal length of the camera, and cx and cy as the coordinates of the optical center.

[0097] When a mass of golden apple snail eggs appears in the field of view of a depth camera, assuming the camera detects the two-dimensional pixel coordinates of the target as (u, v), then the depth information is obtained using the intrinsic parameter matrix k and depth information z of the depth camera. c This allows us to obtain the three-dimensional spatial coordinates of the target object in the camera coordinate system:

[0098]

[0099] Since the relative spatial positions of the camera and the water gun remain unchanged, the camera's extrinsic parameter matrix T can be used to determine this. cw (Through translation and rotation) transform the target object from the camera coordinate system to the water gun coordinate system:

[0100] P w =T cw ·P c

[0101] in It is a 4x4 rigid body transformation matrix; including the rotation matrix R. 3x3 Translation vector t 3x1 .

[0102] Calculate the water gun spray angle

[0103] Based on the three-dimensional spatial coordinates (x) of the target object in the water gun coordinate system w y w , z wThis allows us to calculate the required pitch angle θ (the angle of rotation the water gun needs to be adjusted in the vertical plane) and horizontal angle ψ (the angle of rotation the water gun needs to be adjusted in the horizontal plane) of the gimbal.

[0104]

[0105] B. Control Decisions:

[0106] Error calculation: Based on the target distance fed back by the depth camera and the current angle information provided by the IMU sensor, the system calculates the angle error and distance error between the current water gun pointing and the target. For example, if the target position is within the field of view of the gimbal, but the water flow deviates from the target, the system obtains the error value by calculating the angle between the target and the current direction of the water gun.

[0107] PID algorithm adjustment: The PID controller in the information processing module processes this error and generates a control signal.

[0108] C. Implementation of adjustments:

[0109] Motor-driven gimbal angle adjustment: Based on the control signal generated by the PID controller, the motor is driven to adjust the angle of the gimbal (including horizontal and vertical adjustment) so that the water gun is aimed at the target.

[0110] Dynamic adjustment feedback: The system monitors the direction of the water gun and the distance to the target in real time. If new errors occur (such as the target moving or the direction of the water gun deviating), the control system will recalculate the error and adjust the PID to drive the pan-tilt unit to make further adjustments.

[0111] PWM pump spray control: After obtaining the specific location of the target, the pump will use a PID controller to adjust the duty cycle of the PWM pump in real time according to the combination of delivery distance and head, gradually approaching the ideal range and ensuring that the liquid flow finally hits the target; a high duty cycle (such as 80%-100%) corresponds to high power and high flow rate, which is suitable for long range or high head; a low duty cycle (such as 20%-50%) corresponds to lower power and flow rate, which is suitable for short range.

[0112] If the unmanned surface vessel (USV) fails to identify the egg mass after reaching the designated area, the vessel needs to be readjusted. This may occur if the egg mass is outside the scanning range, or due to factors such as water surface fluctuations or obstructions. Therefore, a mechanism for the USV to make small-scale movements is implemented to ensure maximum coverage of the target area; the specific process is as follows:

[0113] 1. No Egg Mass Detected: If the unmanned surface vessel (USV) fails to detect an egg mass after reaching the target area during secondary identification, the control system will trigger a small-scale movement of the vessel. The USV will make minor adjustments to its position around its original location, ensuring that the camera can cover any potentially missed areas.

[0114] 2. Minor Adjustments: The unmanned vessel will perform minor forward, backward, left, and right turns to ensure the camera scans every possible area of ​​the water surface. If the egg mass is still not identified, it will be identified again.

[0115] 3. Unidentified Egg Mass After Three Adjustments: If the unmanned surface vessel (USV) fails to identify egg masses after three minor adjustments to the target area, it will determine that there are no egg masses in the area or that the egg masses are too scattered. In this case, the USV will proceed to the next target area to continue its operation based on instructions issued from the cloud.

[0116] D. Unmanned Aerial Vehicle (UAV) Re-flight and Secondary Monitoring:

[0117] After the mission is completed, the unmanned surface vessel will return to the starting point along the predetermined path, while the drone will perform a second flight to conduct secondary monitoring. The drone will collect images of the area through high-altitude flight and upload the data to the cloud in real time for egg mass identification and removal effect assessment. The cloud will compare the second flight data with the previous removal records to determine whether the egg masses have been completely removed, and update the risk zoning map of the water area based on the results. If it is found that the egg masses in some areas have not been removed, the cloud will adjust the operation path and priority, and reschedule the unmanned surface vessel to perform the subsequent removal task.

[0118] Example 2:

[0119] A method for predicting the spawning and patrol of golden apple snails based on LSTM (Laminated Threading Mechanism) includes the following steps: Figure 3 As shown,

[0120] S1. Obtain the following data:

[0121] Environmental climate indicators: daily average temperature, daily average humidity, monthly average precipitation, and daily average water temperature;

[0122] Water quality indicators: pH, dissolved oxygen concentration, permanganate index, mineralization, and ammonia nitrogen content; and

[0123] Historical egg-laying intervals of the golden apple snail.

[0124] S2, Data Processing:

[0125] The collected data is normalized and scaled to reduce the data range to [0, 1] to avoid the influence of different units between features. The formula for calculating the normalized value is as follows:

[0126] Where, x min and x max These are the minimum and maximum values ​​of the feature, respectively.

[0127] S3. Model Structure Design:

[0128] The spawning time of the golden apple snail is affected by a variety of factors. To further analyze the impact of environmental climate indicators and water quality indicators on the spawning of the golden apple snail, the prediction process is broken down as follows:

[0129] T next =T base +ΔT season +ΔT eniroment Among them, T next It is the predicted time of the next spawning;

[0130] T base This is a baseline value, assuming no other factors influence the spawning time.

[0131] ΔT season The error is caused by seasonal factors;

[0132] ΔT environnent The error is caused by water quality and environmental factors.

[0133] For the baseline value, we use the mean model for calculation:

[0134] in,

[0135] It is the spawning time observed in historical records;

[0136] N is the total number of historical data.

[0137] Seasonal errors are usually closely related to time (month, season) and temperature, while water quality errors are usually related to multiple physicochemical parameters of the water body (pH, water temperature, dissolved oxygen, ammonia nitrogen concentration, permanganate index). Therefore, we use LSTM to capture the temporal characteristics of seasonal changes and learn the impact of water quality on spawning time.

[0138] The formula for the seasonality error prediction model can be expressed as:

[0139] ΔT season =f season (time features) Among them,

[0140] ΔT season The deviation is caused by seasonal factors (such as temperature, humidity, month, etc.);

[0141] f season It is a seasonal time series function learned by the LSTM model. The input is historical time features (daily average temperature, daily average humidity, monthly average precipitation, and daily average water temperature). LSTM predicts seasonality errors by capturing the long-term dependencies of the time series.

[0142] The specific structure of the LSTM model for seasonal effects is as follows:

[0143] h t =LSTM(x t h t-1 );in:

[0144] x t These are the input features at each time step;

[0145] h t These are the hidden states of the LSTM, reflecting seasonal patterns over a period of time.

[0146] f season The seasonal deviation is ultimately output through a fully connected layer;

[0147] The formula for predicting water quality environmental errors is:

[0148] ΔT environment =f environment (water quality features); among them,

[0149] ΔT environment The deviation is caused by water quality and environmental factors.

[0150] f environment It is an LSTM model. The input is historical water quality data (pH, water temperature, dissolved oxygen, ammonia nitrogen concentration, permanganate index). The LSTM captures the effect of water quality on spawning time.

[0151] The structure of the water quality error LSTM is as follows:

[0152] h t =LSTM(x t h t- 1); where x t It is water quality data; f enviroment Output water quality environmental error

[0153] S4. Outlier Detection and Classification:

[0154] The goal of outlier detection is to identify spawning time predictions that do not conform to normal time series patterns. This design uses an outlier detection method based on Isolation Forest. Isolation Forest is a tree-based ensemble method that effectively detects isolated outliers in data. It gradually "isolates" outlier data points by randomly selecting features and randomly splitting the data, making it highly efficient when processing high-dimensional data.

[0155] S5. Loss Function Design:

[0156] To simultaneously consider the model's prediction accuracy and the impact of outliers, this design employs a weighted loss function, adding a penalty term for outliers to the conventional loss function (such as mean squared error, MSE). In this way, the model not only focuses on optimizing for regular data points but also penalizes potential outlier predictions during training.

[0157] In a conventional LSTM model, we use the mean squared error (MSE) as the loss function, which is calculated as follows:

[0158] Among them, y i

[0159] This is the actual time of spawning. These are the model's predicted values; when dealing with outliers, we need to penalize them more severely. When outliers are detected by the Isolation Forest model, this design introduces a weighting factor to penalize them, making their contribution to the loss function greater.

[0160] The formula for the weighted loss function is:

[0161] in,

[0162] α i =1 for normal data points; α i For outlier data points, the penalty factor α for outliers is set to 10 to make outliers have a greater impact on the loss function.

[0163] Outlier weighting formula:

[0164]

[0165] L1 norm loss (Lasso Regularization):

[0166] To prevent overfitting, this design incorporates L1 norm loss (Lasso Regularization) to further constrain the model's complexity. The L1 regularization loss function is:

[0167] in,

[0168] θ i Here, λ is the model's parameter, and λ is the regularization hyperparameter. This regularization term helps the model maintain sparsity and avoid unnecessary complexity.

[0169] In summary, this design will use a loss function that combines weighted mean squared error (MSE) with L1 regularization:

[0170] in,

[0171] α i λ is the weighting factor, and λ is the regularization coefficient.

[0172] S6. Training Optimization:

[0173] To improve the model's predictive ability, this design uses the Adam optimizer to train the LSTM model. The Adam optimizer can automatically adjust the learning rate to adapt to changes in different parameters, making it particularly suitable for training complex deep learning models.

[0174] The update rules for the Adam optimizer are as follows:

[0175] in,

[0176] θ t These are model parameters, m t and v t These are the mean and variance of the gradient, respectively.

[0177] η is the learning rate, and ∈ is a small constant to prevent division by zero;

[0178] Based on the optimized LSTM model, the next spawning time of the golden apple snail can be predicted. The final predicted time is:

[0179] T next =T base +ΔT season +ΔT environment

[0180] Inspection strategy: By training and predicting using an LSTM model, we can obtain the next spawning interval (T) of the golden apple snail. next This value is a prediction, representing the number of days between the current moment and the next spawning period for the golden apple snail. This prediction takes into full account various influencing factors such as historical data, seasonal variations, and water quality, thus accurately reflecting the spawning cycle of the golden apple snail.

[0181] However, the actual occurrence of spawning can be affected by various unforeseen factors, potentially leading to deviations from model predictions. Therefore, relying solely on prediction results to schedule inspections is not entirely reliable; a certain safety margin needs to be incorporated to ensure the safety and effectiveness of the inspection work. Therefore, the inspection strategy adopted in this design is as follows:

[0182] Safety margin T safety margin Design:

[0183] The training error of the LSTM model is measured using MSE, and a weighting factor α is defined. safety To control the size of the safety margin;

[0184]

[0185] in,

[0186] MSE: Mean squared error of the current model prediction;

[0187] MSE max The largest mean square error in history, used as the basis for standardization;

[0188] ∈: a small constant to prevent the denominator from being zero;

[0189] To better consider the baseline value T base The role of introducing a weighting factor β base To compare the differences between the baseline and the predicted values;

[0190] in,

[0191] T next The next spawning time predicted by the model;

[0192] T base : Baseline value, assuming no other factors affect the spawning time;

[0193] ∈: a small constant to prevent the denominator from being zero;

[0194] β base-max : For β base The maximum value is set to prevent excessive impact on the inspection interval when the prediction error is too large. When the prediction is relatively accurate, it is set to 0.2 to 0.5; when the prediction is unstable, it is set to 0.5 to 1.0.

[0195] Combining α safety With β base Safety margin T safetymargin The final formula is designed as follows:

[0196] T safety margin =T next ×(1-α safety ×MSE)+β base ×|T base -T next |;

[0197] If the predicted value is T next The inspection time interval should be: T inspection =min(T)next T base T safety margin );in,

[0198] T next The next spawning time predicted by the model;

[0199] T base Baseline value: the time of spawning assuming no other influencing factors.

[0200] T safety margin Based on the uncertainty of the prediction, it represents a safety margin for the prediction time interval.

[0201] Through the above design, the patrol time interval (T) inspection It can provide a flexible and safe time window for carrying out inspection tasks while taking into account the spawning time. For each predicted spawning cycle, the model dynamically adjusts the inspection time nodes according to the actual environmental and water quality conditions.

[0202] If T next A shorter interval means that the predicted next spawning cycle is shorter, and the inspection interval will be more frequent to ensure that the inspection personnel can arrive at the site in a timely manner for inspection.

[0203] If T base The longer the period, the more flexible the patrol intervals, as historical data shows that the golden apple snail has a relatively long spawning cycle. Patrol personnel can appropriately postpone their patrol tasks.

[0204] Safety margin T safety margin Its function is to ensure that the inspection time is not too delayed, thus avoiding the risk of missed inspections due to changes in the external environment or errors in data prediction.

[0205] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for monitoring and controlling golden apple snail egg masses using a low-altitude unmanned aerial vehicle (UAV) and an unmanned surface vessel (USV), characterized in that, Includes the following steps: S1: Use drones to collect images and locate coordinates of golden apple snail egg masses in the monitored water area to construct a preliminary distribution map of golden apple snail egg masses; S2: The coordinates are then divided using the DBSCAN clustering method, thereby dividing the monitored water area into zones with different ecological risk levels; S3: Based on the ecological risk level area division information, set up a density hotspot priority list for the Golden Apple Snail egg mass removal task, prioritize the handling of high ecological risk emergency areas with high Golden Apple Snail egg mass density, and combine the weather, water flow, and obstacle factors in the monitored water area to complete the unmanned vessel route planning for Golden Apple Snail egg mass removal. S4: The unmanned vessel accurately and thoroughly removes the golden apple snail egg masses in the monitored waters according to the removal route; S5: After the unmanned surface vessel (USV) completes its initial cleanup of the monitored waters, it returns to its starting point to await further instructions. The USV then re-flys at high altitude to collect images of the monitored waters and uploads the image data to the cloud in real time for egg mass identification and cleanup effectiveness assessment. The cloud compares the re-fly data with previous cleanup records to determine if the egg masses have been completely removed and updates the ecological risk zoning map of the waters based on the results. If it finds that egg masses have not been removed in certain areas, the cloud will adjust the operational path and priority, and reassign the USV to perform the cleanup task again until all cleanup tasks are completed. Image acquisition in S1 includes the UAV first flying at low altitude along the monitored water area, completing continuous environmental mapping and acquiring images of golden apple snail egg masses using SLAM, and then ascending to high altitude to acquire global images of the monitored area; wherein... During the acquisition of images of golden apple snail egg masses, YOLOv5 image recognition was used in the cloud to detect, identify, and remove impurities from the egg masses, resulting in a monitoring and positioning coordinate map of the egg masses; the NWD loss function was added to the YOLOv5 image recognition; and... The risk level classification is as follows: After the DBSCAN clustering method is completed, the density values ​​of golden apple snail egg masses are statistically analyzed for several clusters obtained by clustering. The clusters are sorted according to the density values ​​of golden apple snail egg masses. The 80% quantile is selected as the critical value for high ecological risk area; the 40% quantile is selected as the critical value for medium ecological risk area; and areas with less than 40% quantile are considered as low ecological risk areas.

2. The method for monitoring and controlling golden apple snail egg masses in collaboration with low-altitude unmanned aerial vehicles and unmanned surface vessels according to claim 1, characterized in that, The low-altitude range is 8 to 20 meters; the high-altitude range is 30 to 80 meters.

3. The method for monitoring and controlling golden apple snail egg masses in collaboration with low-altitude unmanned aerial vehicles and unmanned surface vessels according to claim 2, characterized in that, In S4, when the unmanned surface vessel (USV) arrives at the designated removal area, it first needs to perform a secondary location confirmation of the golden apple snail egg masses in that area. If the USV egg masses are found to be consistent with the corresponding positions on the monitoring and positioning coordinate map, the USV egg masses are removed by high-speed, high-pressure water jets from the water guns mounted on the USV. If they are inconsistent, the position of the USV needs to be fine-tuned until the corresponding USV egg masses are identified, and then removal is performed. If the corresponding USV egg masses still cannot be identified after multiple fine-tunings, the USV moves to the next removal area and repeats the above steps until the removal task is completed.

4. The method for monitoring and controlling golden apple snail egg masses in collaboration with low-altitude unmanned aerial vehicles and unmanned surface vessels according to claim 3, characterized in that, The removal of golden apple snail egg masses includes information acquisition, control decision-making, and execution adjustment. Information acquisition includes collecting the location information of the target golden apple snail egg masses and the water gun spray angle information. Control decision-making includes the angular error and distance error between the water gun pointing and the target. The PID controller in the information processing module processes the above two types of errors and generates control signals. Execution adjustment includes driving the motor to adjust the horizontal and vertical angles of the gimbal according to the control signals, dynamically adjusting feedback, and regulating the PWM water pump spray.

5. The method for monitoring and controlling golden apple snail egg masses in collaboration with a low-altitude unmanned aerial vehicle and an unmanned surface vessel according to claim 4, characterized in that, Obtain the location information of the target golden apple snail egg mass and the water jet angle information: With the camera as the origin, the three-dimensional spatial coordinates of the target golden apple snail egg mass relative to the camera are P. c =(x c y c , z c ); With the water gun as the origin, the three-dimensional spatial coordinates of the target golden apple snail egg mass relative to the water gun are P. w =(x w ,y w , z w ); camera intrinsic parameter matrix Among them, f x and f y y is the focal length of the camera, and cx and cy are the coordinates of the camera's optical center. When a mass of golden apple snail eggs appears in the field of view of a depth camera, assuming the camera detects the two-dimensional pixel coordinates of the target as (u, v), then the depth information is obtained using the intrinsic parameter matrix k and depth information z of the depth camera. c This allows us to obtain the three-dimensional spatial coordinates of the target object in the camera coordinate system: Since the relative spatial positions of the camera and the water gun remain unchanged, the camera's extrinsic parameter matrix T can be used to determine this. cw Transform the target object from the camera coordinate system to the water gun coordinate system: P w =T cw ·P c ;in, in It is a 4x4 rigid body transformation matrix; including the rotation matrix R. 3x3 Translation vector t 3x1 ; Based on the three-dimensional spatial coordinates (x) of the target object in the water gun coordinate system w y w ,z w ), that is, P w It can calculate the pitch angle θ that the gimbal needs to be adjusted. That is, the rotation angle and horizontal angle that the water gun needs to be adjusted in the vertical plane. This refers to the angle of rotation that the water gun needs to be adjusted in the horizontal plane.

6. The method for monitoring and controlling golden apple snail egg masses in collaboration with a low-altitude unmanned aerial vehicle and an unmanned surface vessel according to claim 1 or 5, characterized in that, A growth cycle prediction mechanism for golden apple snail egg masses is superimposed on the aforementioned density hotspot priority list.

7. The method for monitoring and controlling golden apple snail egg masses in collaboration with a low-altitude unmanned aerial vehicle and an unmanned surface vessel as described in claim 6, characterized in that, The proposed golden apple snail egg mass growth cycle prediction mechanism uses an LSTM model to capture the egg-laying patterns of the golden apple snail by analyzing historical environmental and climate data, historical water quality data, and historical egg-laying time intervals within the monitoring area, thereby predicting the time of the next egg-laying.

8. The method for monitoring and controlling golden apple snail egg masses in collaboration with a low-altitude unmanned aerial vehicle and an unmanned surface vessel according to claim 7, characterized in that, The growth cycle of the golden apple snail egg mass can be predicted using the following formula: T next =T base +ΔT season +ΔT environment ;in, T next It is the predicted time of the next spawning; T base This is a baseline value, assuming no other factors affect the spawning time; ΔT season The error is caused by seasonal factors; ΔT environment The error is caused by water quality and environmental factors; For the baseline value, we use the mean model for calculation: in, N represents the spawning time observed in historical data; N is the total amount of historical data.

9. The method for monitoring and controlling golden apple snail egg masses in collaboration with a low-altitude unmanned aerial vehicle and an unmanned surface vessel as described in claim 8, characterized in that, This also includes the safety margin T. safetyrnargin : The training error of the LSTM model is measured using MSE, and a weighting factor α is defined. safety To control the size of the safety margin; in: MSE: Mean squared error of the current model prediction; MSE max The largest mean square error in history, used as the basis for standardization; ∈: a small constant to prevent the denominator from being zero; To better consider the baseline value T base The role of introducing a weighting factor β base To compare the difference between the baseline value and the predicted value: in, T next The next spawning time predicted by the model; T base : Baseline value, assuming no other factors affect the spawning time; ∈: a small constant to prevent the denominator from being zero; β base-max : For β base The maximum value is set to prevent excessive influence on the inspection interval when the prediction error is too large; when the prediction is relatively accurate, it is set to 0.2 to 0.5; when the prediction is unstable, it is set to 0.5 to 1.

0. Combining α safety With β base Safety margin T safety margin The final formula is designed as follows: T safety margin =T next ×(1-α safety ×MSE)+β base ×|T base -T next |。 10. The method for monitoring and controlling golden apple snail egg masses in collaboration with a low-altitude unmanned aerial vehicle and an unmanned surface vessel according to claim 9, characterized in that, This also includes the design of inspection time intervals: If the predicted value is T next The inspection interval should then be: T inspection =min(T) next T base T safety margin ),in: T next The next spawning time predicted by the model; T base : Baseline value, assuming no other factors affect the spawning time; T safety margin Based on the uncertainty of the prediction, it provides a safety margin for the prediction time interval.

Images