An air-ground cooperative agricultural patrol system and method based on visual perception and intelligent path planning

Abstract

The application provides a land-air collaborative agricultural patrol system and method based on vision and path planning, which comprises a path planning module and a data analysis module running in a cloud computing center, a collaborative control module and a target identification module running on an edge computing node, and an image acquisition module running on a terminal device; the terminal device comprises a robot dog, a drone and a camera, which are used for collecting farmland images from the ground at close range and local monitoring, collecting farmland images from the air in a wide range and large-scale patrol, and continuously collecting farmland images from fixed points, and converting light signals of farmland and its surrounding environment images into first image information; the edge computing node is used for receiving and processing the first image information from the terminal device and data from the cloud computing center; and the cloud computing center is used for performing global scheduling and data analysis, solving the problems of low bird and beast invasion identification precision and limited coverage in agricultural monitoring, and improving the intelligent level and efficiency of agricultural monitoring.

Description

Technical Field

[0001] This invention relates to the fields of smart agriculture and agricultural information technology, specifically to a land-air collaborative agricultural patrol system and method based on visual perception and intelligent path planning. More particularly, this invention relates to a monitoring and early warning system that enables 24 / 7, accurate identification and intelligent response to threats such as pests, diseases, and animal intrusions in large-scale farmland through the collaborative operation of cloud computing centers, edge computing nodes, and terminal devices (including robot dogs, drones, and fixed cameras). Background Technology

[0002] The development of smart agriculture has promoted the intelligent and information-based management of agricultural production processes. As a key link in ensuring crop growth and yield, the technological evolution of farmland inspection has attracted much attention. Existing agricultural monitoring systems mostly rely on single technologies or equipment to achieve inspection functions, such as fixed cameras, drones, or ground robots. However, these solutions have significant shortcomings in terms of recognition accuracy, coverage, real-time performance, and collaborative control.

[0003] In the field of single-track planning technology for UAV-based inspections, Jiangsu University's invention patent, "A Real-Time Planning Method for UAV Agricultural Inspection Trajectory Based on Images" (Publication No.: CN108827297A), achieves real-time UAV trajectory planning by integrating image recognition with inertial devices (such as magnetometers, gyroscopes, microwave radar, and accelerometers). This method extracts the planting area boundary based on crop color and brightness features, estimates the relative altitude between the UAV and crops using Kalman filtering, and maintains a stable heading angle to generate an inspection path. However, this solution only addresses a single UAV platform, does not involve coordinated control of land and air equipment, and lacks the ability to perform fine-grained monitoring of local ground areas, making it unable to cope with multi-layered threats in complex farmland environments. Furthermore, its path planning relies on pre-set sensor data, making it difficult to dynamically adjust the task based on real-time recognition results, resulting in a lag in response speed.

[0004] To address the need for large-scale farmland inspections, existing research has proposed multi-UAV collaborative task allocation and path planning techniques. For example, an improved ant colony optimization (ACO) algorithm for agricultural inspection scenarios optimizes task allocation and path search for multiple UAVs through grid map preprocessing, feature point extraction, and non-uniform pheromone distribution. While this method improves patrol efficiency and convergence speed, it focuses on aerial platforms and fails to integrate ground equipment resources, resulting in insufficient short-range monitoring capabilities. Furthermore, these methods rely on centralized scheduling to respond to dynamic anomalies (such as sudden bird or animal intrusions), limiting their real-time replanning capabilities and making it difficult to achieve full-coverage closed-loop control.

[0005] Furthermore, a recent study published in the *ACM Journal on Autonomous Transportation Systems* proposed a collaborative perception and constrained coverage framework for multi-UAVs in farmland monitoring. This framework optimizes task allocation and trajectories for heterogeneous UAVs through region decomposition and backward-level mixed integer linear programming (MILP). While this approach considers the physical constraints and data acquisition needs of the UAVs, it remains limited to aerial platform clusters and does not incorporate ground mobile nodes (such as robotic dogs) to form a three-dimensional monitoring network. Moreover, this type of method suffers from insufficient integration of edge computing nodes, resulting in high closed-loop latency in image recognition and path planning, which affects the efficiency of real-time handling of targets such as pests and diseases.

[0006] In summary, while existing agricultural inspection technologies have made progress in path planning for single devices or platforms, they still suffer from the following common bottlenecks: insufficient collaboration, with most solutions focusing on independent applications of drones or ground agricultural machinery, lacking a mechanism for task collaboration and data fusion between land and air equipment, resulting in blind spots in monitoring coverage; weak dynamic response capabilities, as path planning is mostly based on static maps or preset routes, leading to low efficiency in replanning when faced with real-time identified anomalies such as pests, diseases, or animal intrusions, and failing to form a closed-loop control with edge computing nodes; and a contradiction between recognition accuracy and real-time performance, as the limited field of view of sensors on a single platform makes target recognition in complex environments susceptible to interference from lighting and occlusion, while the centralized processing mode of cloud computing centers introduces latency, making it difficult to meet the real-time requirements of 24-hour uninterrupted inspection. Therefore, there is an urgent need in this field for a collaborative inspection system that can integrate cloud computing, edge computing, and terminal devices (such as robot dogs, drones, and cameras) to achieve accurate identification and rapid response to threats to farmland through global path planning, real-time image recognition, and dynamic task allocation. Summary of the Invention

[0007] This invention addresses the problems of insufficient coordination, weak dynamic response capabilities, low recognition accuracy, and limited coverage in existing agricultural patrol systems. It provides a land-air collaborative agricultural patrol system and method based on vision and path planning. Through the collaborative operation of a cloud computing center, edge computing nodes, and terminal devices (including robotic dogs, drones, and fixed cameras), this system achieves 24 / 7 uninterrupted, accurate identification and intelligent response to threats such as farmland pests, diseases, and animal intrusions, effectively improving the intelligence level and operational efficiency of agricultural monitoring.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a land-air collaborative agricultural patrol system based on vision and path planning, comprising a path planning module and a data analysis module running in a cloud computing center, a collaborative control module and a target recognition module running on edge computing nodes, and an image acquisition module running on a terminal device; characterized in that: The terminal equipment includes a robot dog, a drone, and a camera, wherein: The image acquisition module on the robot dog is used to acquire images of farmland from close range on the ground and for local monitoring; The image acquisition module on the drone is used to acquire wide-area images of farmland from the air and to conduct large-scale inspections; The image acquisition module on the camera is used to continuously acquire images of farmland from fixed locations; The image acquisition module is connected to the edge computing node and includes: an optical image sensing camera, a buffer register, and a network transmitter; wherein: An optical image sensing camera is used to capture images of farmland and its surrounding environment, and to convert the light signals of the images of farmland and its surrounding environment into first image information. The buffer register is used to store the first image information; The network transmitter is used to transmit the first image information stored in the buffer register to the edge computing node; The edge computing node is connected to both the terminal device and the cloud computing center, and is used to receive and process first image information from the terminal device and data from the cloud computing center. It includes: a collaborative control module and a target recognition module, wherein: The collaborative control module receives and parses global path instructions from the cloud computing center, generates specific action control signals based on the real-time context, and distributes them to the corresponding robot dog, drone, and camera on the terminal devices. It includes: a land control unit, an air control unit, and a camera control unit, wherein: The land control unit is used to control the robot dog's behavior. It performs dynamic path planning and local obstacle avoidance based on global path instructions in order to reach the designated location for inspection. The airborne control unit is used to control the behavior of the drone. It parses the global path instructions into specific flight waypoints and flight altitudes, and directs the drone to complete fixed-point area scanning and inspection of a large area of ​​farmland. The camera control unit is used to control cameras at fixed locations. Based on global commands and collaborative strategies, it generates control signals, including pan-tilt rotation angle and preset position switching, to adjust the monitoring field of view. The target recognition module analyzes and recognizes the first image information uploaded by the image acquisition module, extracts feature information from the image, and compares it with a pre-set agricultural threat model. It includes: an image preprocessing unit, a feature extraction unit, and a classification and recognition unit, wherein: The image preprocessing unit is used to standardize the first image information, and its operations include image denoising, contrast enhancement and scale normalization to form the second image information. The feature extraction unit is used to extract key visual features from the preprocessed second image information. Based on the convolutional neural network model, it automatically learns and extracts multi-scale visual feature vectors related to agricultural threats in the image. The multi-scale visual feature vectors include, but are not limited to, the morphological texture of pests, the mottled color patches of diseased leaves, and the outline and posture of invasive birds and animals. The classification and recognition unit is used to make the final judgment of the target based on the extracted multi-scale visual feature vector. It inputs the multi-scale visual feature vector into the preset classifier for inference calculation, and then outputs the structured recognition result including the target category, location bounding box and confidence level. It compares the result with the preset model in real time to complete the recognition of specific pests or bird and animal invasion targets. The cloud computing center is used to execute global scheduling and data analysis tasks, and includes: a path planning module and a data analysis module; wherein: The path planning module generates globally optimal patrol paths for terminal devices. Based on farmland geographic information, historical patrol data, and real-time task requests, it formulates a land-air collaborative patrol plan covering the entire monitoring area. It includes: a global path unit, a task allocation unit, and a dynamic adjustment unit; wherein: The global path unit is used to generate the basic land movement path of the robot dog and the basic flight route of the drone based on map information and mission objectives using the Dijkstra algorithm. The task allocation unit is used to assign specific patrol tasks and areas to the most suitable terminal devices based on the device status and location of the robot dog, drone, and camera, so as to achieve load balancing and optimal efficiency. The dynamic adjustment unit is used to receive abnormal alarms from edge computing nodes and perform real-time global path replanning for the affected area, directing terminal tasks to respond to emergencies. The data analysis module is used for in-depth analysis and model optimization of the global data collected by the system. It receives and integrates structured identification results from edge computing nodes, status information from terminal devices, and external environmental data to make comprehensive decisions. It includes: a data fusion unit and a decision generation unit; wherein: The data fusion unit is used to perform spatiotemporal alignment and correlation of multi-source heterogeneous data to construct a unified situational view that includes image recognition results, device location, and environmental factors. The decision generation unit is used to perform trend analysis and prediction based on the global data fused by the data fusion unit, and to generate high-level decision information, including early warning of pest and disease outbreak risks, suggestions for strengthening monitoring in key areas, and the generation of periodic inspection reports.

[0009] Furthermore, the global path unit of the path planning module in the cloud computing center first executes the default patrol route. This default patrol route is a default patrol route covering the entire domain calculated using Dijkstra's algorithm, and its total path cost C_total is calculated based on weight coefficients. Euclidean distance The specific calculation formula is as follows: In the formula, Indicates from coordinate point to coordinate point The weighting coefficients are: α is the terrain complexity coefficient, β is the historical threat coefficient, and γ is the equipment energy consumption coefficient. Represents the coordinates of the node. to coordinate point The Euclidean distance, Σ, represents the sum of the costs of all edges along the entire path p. The cost is the shortest path from the starting point to node j. The task allocation unit of the path planning module in the cloud computing center allocates a suitable terminal device to each newly generated patrol task based on the following decision function S(d,t): In the formula, S(d,t) represents the fitness score of device d for task t. E(d) represents the Euclidean distance from the current coordinate point d to the task coordinate point t, and E(d) represents the remaining battery power of device d. α2, β2, γ2 are the normalized weight coefficients of each factor; the task allocation unit solves the above optimization problem and allocates the task to the terminal device that maximizes the total adaptability score. The dynamic adjustment unit of the path planning module in the cloud computing center continuously monitors the target information received by the data analysis module during operation. When it receives a pest or disease identification alarm or animal intrusion alarm generated by the target identification module on the edge computing node, it loads predefined response weights Wa and Wb according to the target category and compares them with the default weight W to trigger a differentiated response strategy. The dynamic adjustment unit of the path planning module in the cloud computing center triggers a differentiated response strategy based on the default weight W value: If the identified target is a pest or disease, then when Wa < W, the dynamic adjustment unit executes the recording strategy, that is, records the coordinate, time, and type information of the occurrence of the pest or disease to the decision-making generation unit of the cloud computing center; If the identified target is the invasion of birds and beasts, then when Wb > W, the dynamic adjustment unit executes the expulsion strategy. First, it records the invasion coordinates, and then immediately sends a high-priority expulsion task to the task allocation unit; the task allocation unit calculates the terminal device with the nearest location and the fastest response in the current state according to the right decision function S(d, t), and assigns it to go to this coordinate; finally, the global path unit plans an optimal path for the assigned device to go to the invasion coordinate, and performs real-time optimization of the path by the gradient descent method, where represents the path point of the k-th iteration, and η is the learning rate.

[0010] Furthermore, the data fusion unit of the data analysis module of the cloud computing center uses the Kalman filter algorithm to fuse and perform state estimation on multi-source heterogeneous data from edge computing nodes, and constructs a unified global situation view. The fusion process is as follows: where, is the state prediction value, is the state transition matrix, is the prediction covariance, is the process noise covariance, is the Kalman gain, is the observation vector, is the observation matrix, is the observation noise covariance, and are the updated state estimation and covariance respectively; The decision-making generation unit of the data analysis module of the cloud computing center, based on the fused global data, performs risk scoring, trend prediction, and spatial propagation modeling, and generates decision-making information including early warning, key monitoring suggestions, and inspection reports according to the output results. It specifically includes the following steps: (1) Calculate the agricultural threat risk score based on the multi-dimensional feature vector XX and the weight vector ΛΛ: (2) Use the autoregressive integrated moving average model to model and predict the time series data of pests and diseases: where, is the lag operator, is the time series observation value, is the model parameter, is the white noise; (3) Modeling the spatial spread trend of pests and diseases based on the reaction-diffusion equation: in, For pest and disease density, Let α be the diffusion coefficient and α be the natural growth rate. β represents the environmental capacity, and β represents the control intensity. Based on the output of the above model, the decision generation unit automatically generates decision recommendations including risk level, predicted trend, and propagation path, and triggers corresponding early warning or control instructions. Furthermore, the image preprocessing unit of the target recognition module on the edge computing node standardizes the first image information uploaded by the image acquisition module on the terminal device to form second image information. The processing includes: Gaussian filtering noise reduction operation: in, For the first image information, For Gaussian kernel function, Standard deviation; Histogram equalization enhancement operation: in, For gray levels, grayscale The number of pixels, Image size; The feature extraction unit of the target recognition module on the edge computing node extracts multi-scale visual features from the preprocessed second image information based on a convolutional neural network. Its calculation process includes: Convolutional feature mapping: in, For the feature map of layer l, For convolution kernel weights, For bias terms, It is the ReLU activation function; Global average pooling: in, For the final feature vector, For feature dimensions; The classification and recognition unit of the target recognition module on the edge computing node completes the final recognition and localization of the target based on the extracted feature vectors. Its calculation process includes: Target classification and confidence calculation: in, Let be the posterior probability of class c, be the predicted class label, and be the confidence level. Target bounding box regression: in, The coordinates of the target center are This refers to the bounding box dimensions; World coordinate system transformation: in, For the camera intrinsic parameter matrix, For depth information, This represents the camera's position in the world coordinate system.

[0011] Furthermore, the land control unit of the collaborative control module of the edge computing node performs motion control on the robot dog of the terminal device based on the global path information generated by the path planning module of the cloud computing center. Its calculation process includes: Path cost control: The land control unit of the collaborative control module of the edge computing node receives global path instructions and path cost information, including the minimum path cost, from the path planning module of the cloud computing center. It is used to guide the robot dog's local movement decisions; PID motion control: in, This represents the pose error between the robot dog's current position and the target position in the global path command. These are the proportional, integral, and differential coefficients, respectively. Kinematic control model: in, These are the robot dog's linear velocity and angular velocity, respectively, which are adjusted in real time based on the path point sequence output by the path planning module. The airborne control unit of the collaborative control module of the edge computing node performs flight control based on the UAV dynamics model and wind resistance compensation strategy of the terminal device. Its calculation process includes: Dynamic equations: in, For the quality of drones, For position vectors, For rotation matrix, The lift vector, Let ω be the moment of inertia, ω be the angular velocity, and τ be the torque. Quaternion gesture representation: Heading angle wind compensation: in, The compensated heading angle, For wind speed, Wind direction; The camera control unit of the collaborative control module of the edge computing node generates control signals based on a preset positioning strategy and a field-of-view optimization model, including: Preset position switching control: Based on global commands and collaborative strategies, the system automatically switches to the preset monitoring position and adjusts the monitoring field of view to cover key areas. The preset position is due north with an elevation angle of 60 degrees. Gimbal rotation control: Based on the target location information, the pan-tilt-zoom (PTZ) rotation angle is calculated to ensure the target remains centered in the monitoring field of view. The formula for generating the control signal is as follows: In the formula Using the target world coordinates, Location for camera installation; Optimized field of view coverage: in, For a moment Surveillance coverage Assuming energy consumption costs, α and β are weighting coefficients. To maximize monitoring coverage, the collaborative monitoring strategy of multiple cameras is dynamically adjusted.

[0012] Furthermore, the optical image sensing camera of the image acquisition module of the terminal device is used to acquire raw image data of the farmland environment, and generates first image information for use by the target recognition module through the following processing: Image acquisition and digital processing are performed according to the following formula: in, The raw light signal collected by the sensor. The output digital image after discrete cosine transform. These are the transformation coefficients; The optical image sensing camera also outputs embedded metadata, including: acquisition timestamp t, device location coordinates. Camera intrinsic parameter matrix K, depth information d; Embedded metadata and Together they constitute the first image information .

[0013] The cloud computing center is a conventional server.

[0014] The edge computing node is a conventional computer.

[0015] Another objective of this invention is to provide an operational method for conducting land-air collaborative agricultural patrols using a vision-based and path planning-based land-air collaborative agricultural patrol system.

[0016] The operation method of the land-air coordinated agricultural patrol system based on vision and path planning of the present invention for land-air coordinated agricultural patrol includes the following steps: S1: System initialization; The cloud computing center loads the pre-stored farmland geographic information map, historical inspection data, and pre-set agricultural threat model, and obtains the real-time status information and location of all terminal devices through edge computing nodes. The terminal devices include robot dogs, drones, and cameras. S2: Global Path Planning and Task Allocation; The path planning module of the cloud computing center, based on the pre-stored farmland geographic information map and historical patrol data loaded in S1, uses the Dijkstra algorithm through its global path unit to calculate the basic land travel path of the robot dog and the basic flight route of the drone covering the entire monitoring area, and generates global path instructions; Then, the task allocation unit of the path planning module of the cloud computing center, according to the real-time status information and location of the terminal devices obtained in S1, assigns the specific patrol area and tasks to the corresponding robot dog, drone and camera, forming a global patrol plan that coordinates land and air. S3: Cooperative control command generation and distribution; the cooperative control module of the edge computing node receives the global path command generated by the path planning module in S2 and parses the command: The land control unit of the collaborative control module generates motion control signals for the robot dog, including linear velocity and angular velocity, based on the global path instructions and the robot dog's real-time pose. The airborne control unit of the collaborative control module parses the global path command into UAV flight control signals containing specific flight waypoints, altitudes, and attitudes; The camera control unit of the collaborative control module generates camera control signals for adjusting the gimbal rotation angle and switching preset positions according to the collaborative strategy in the global path instructions; the collaborative control module on the edge computing node sends the generated motion control signals, flight control signals and camera control signals to the corresponding robot dog, drone and camera respectively; S4: Farmland environment image acquisition; The terminal device executes the control signal sent by S3 and synchronously acquires farmland environment images through its image acquisition module: The robot dog collects close-up images of the ground while moving on land; The drone acquires wide-area aerial images during flight; The camera continuously captures images from a fixed location; The image acquisition module will send the acquired first image information, which includes raw image data and embedded metadata, to the edge computing node through its network transmitter. S5: Image recognition and threat target detection; the target recognition module of the edge computing node, which receives and processes the first image information from S4. The image preprocessing unit of the target recognition module performs normalization processing on the first image information, including Gaussian filtering for noise reduction and histogram equalization enhancement, to form the second image information; The feature extraction unit of the target recognition module automatically extracts multi-scale visual feature vectors related to agricultural threats from the second image information based on a convolutional neural network model; The classification and recognition unit of the target recognition module inputs the extracted multi-scale visual feature vectors into a preset classifier for inference calculation, and then outputs a structured recognition result containing the target category, location bounding box and confidence level, thus completing the recognition of specific pests or invasive bird and animal targets. S6: Upload the recognition results; The edge computing node uploads the structured recognition results obtained from the target recognition module in S5 to the data analysis module of the cloud computing center; S7: Multi-source data fusion and situational awareness construction; The data fusion unit of the data analysis module in the cloud computing center receives structured recognition results from S6, real-time status information from terminal devices, and external environmental data. It then uses the Kalman filter algorithm to perform spatiotemporal alignment and fusion processing on these multi-source heterogeneous data to construct a unified global situational awareness view. S8: Data Analysis and Decision Generation; the decision generation unit of the data analysis module in the cloud computing center, performing the following in-depth analysis based on the fused global situational view data from S7: Calculate agricultural threat risk scores; An autoregressive integral moving average model was used to predict the time series data of pests and diseases. Based on the reaction-diffusion equation, the spatial spread trend of pests and diseases is modeled; Based on the above analysis results, high-level decision-making information is generated, including early warning of pest and disease outbreak risks, recommendations for strengthening monitoring in key areas, and periodic inspection reports; S9: Anomaly Response and Dynamic Path Adjustment; The dynamic adjustment unit of the path planning module in the cloud computing center continuously monitors the decision information generated by the data analysis module in S8 and the recognition results uploaded in S6; when an anomaly alarm generated by the target recognition module in S5 is received, a dynamic response strategy is triggered: If the identified target is a pest or disease, the dynamic adjustment unit executes the recording strategy and records the location, time, and type information of the event to the decision generation unit in S8. If the target is identified as an intrusion by birds or animals, the dynamic adjustment unit executes a deportation strategy: first, it records the intrusion coordinates, and then immediately sends a high-priority deportation task to the task allocation unit; the task allocation unit allocates the device with the closest location and fastest response from the terminal devices to execute the deportation task based on the decision function S(d,t); the global path unit then plans an optimal path to the intrusion coordinates for the device. S10: Collaborative control closed-loop execution; The collaborative control module of the edge computing node receives the new path instructions generated after dynamic adjustment in S9, and repeats the process of S3 to generate new control signals and send them to the corresponding terminal devices, instructing the terminal devices to perform drive-away tasks or to strengthen monitoring of key areas, thus forming closed-loop control. S11: System continuous operation and optimization; the system repeats steps S4 to S10 to achieve 24-hour uninterrupted land-air coordinated agricultural patrol.

[0017] Compared with existing technologies, this invention has the following advantages and effects: First, by constructing a land-air collaborative terminal system composed of a robot dog, drones, and fixed cameras, it achieves three-dimensional stereoscopic image acquisition at close range on the ground, wide area in the air, and continuous fixed-point imaging. This fundamentally solves the problem of limited monitoring perspective and blind spots on a single platform, significantly expanding the effective patrol coverage. Second, relying on the collaborative architecture of "global optimization of cloud computing center + real-time processing of edge computing nodes," time-consuming image recognition and rapid response tasks are pushed to the edge, greatly reducing data processing latency and ensuring real-time accurate identification and rapid alarm of dynamic targets such as pests, birds, and animals. Third, by introducing a global path planning algorithm that integrates multi-dimensional costs and a task allocation model based on real-time status, the system can intelligently schedule different terminal devices, achieving load balancing while ensuring optimal efficiency in patrol task execution. Fourth, the system has the ability to dynamically adjust based on real-time recognition results. In particular, it can automatically trigger a rapid response mechanism for emergency events such as bird and animal intrusions, replanning the path and assigning the nearest device to drive them away, greatly improving the intelligence level of proactive defense and emergency response. Finally, the cloud computing center, through deep fusion and analysis of multi-source heterogeneous data, has achieved a leap from single-event detection to regional risk prediction and propagation trend assessment, providing agricultural producers with forward-looking decision support and thus comprehensively improving the proactive early warning capabilities and overall management efficiency of agricultural inspections. In summary, this invention not only effectively solves the problems of low identification accuracy, limited coverage, and delayed response in traditional agricultural monitoring, but also significantly improves the intelligence level, adaptability, and overall operational efficiency of agricultural monitoring systems through multi-level technological collaboration and closed-loop optimization. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the system modules of the present invention; Figure 2 This is a flowchart illustrating the specific execution process of the system of the present invention; Figure 3 This is a view of the overall situation of farmland. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer and more complete, the invention will be further described below with reference to specific embodiments 1, 2, and 3, wherein: Example 1: No anomalies were found, and all devices maintained their default execution state for inspection. Example 2: An anomaly is detected, identified as pests or diseases, and the coordinates are marked. The patrol is maintained in its default state without executing the expulsion task. Example 3: An anomaly was detected, indicating an invasion by birds or animals. The coordinates were marked, and a task to drive them away was carried out.

[0020] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. Other systems and methods of this invention will become apparent to those skilled in the art after reviewing the following detailed description; all such additional systems, methods, features and advantages are intended to be included within the scope of protection of this invention.

[0021] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or component referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present patent. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0022] Example The present invention provides a land-air collaborative agricultural patrol system based on vision and path planning, comprising a path planning module and a data analysis module running in a cloud computing center, a collaborative control module and a target recognition module running on edge computing nodes, and an image acquisition module running on a terminal device; characterized in that: The terminal equipment includes a robot dog, a drone, and a camera, wherein: The image acquisition module on the robot dog is used to acquire images of farmland from close range on the ground and for local monitoring; The image acquisition module on the drone is used to acquire wide-area images of farmland from the air and to conduct large-scale inspections; The image acquisition module on the camera is used to continuously acquire images of farmland from fixed locations; The image acquisition module is connected to the edge computing node and includes: an optical image sensing camera, a buffer register, and a network transmitter; wherein: An optical image sensing camera is used to capture images of farmland and its surrounding environment, and to convert the light signals of the images of farmland and its surrounding environment into first image information. The buffer register is used to store the first image information; The network transmitter is used to transmit the first image information stored in the buffer register to the edge computing node; The edge computing node is connected to both the terminal device and the cloud computing center, and is used to receive and process first image information from the terminal device and data from the cloud computing center. It includes: a collaborative control module and a target recognition module, wherein: The collaborative control module receives and parses global path instructions from the cloud computing center, generates specific action control signals based on the real-time context, and distributes them to the corresponding robot dog, drone, and camera on the terminal devices. It includes: a land control unit, an air control unit, and a camera control unit, wherein: The land control unit is used to control the robot dog's behavior. It performs dynamic path planning and local obstacle avoidance based on global path instructions in order to reach the designated location for inspection. The airborne control unit is used to control the behavior of the drone. It parses the global path instructions into specific flight waypoints and flight altitudes, and directs the drone to complete fixed-point area scanning and inspection of a large area of ​​farmland. The camera control unit is used to control cameras at fixed locations. Based on global commands and collaborative strategies, it generates control signals, including pan-tilt rotation angle and preset position switching, to adjust the monitoring field of view. The target recognition module analyzes and recognizes the first image information uploaded by the image acquisition module, extracts feature information from the image, and compares it with a pre-set agricultural threat model. It includes: an image preprocessing unit, a feature extraction unit, and a classification and recognition unit, wherein: The image preprocessing unit is used to standardize the first image information, and its operations include image denoising, contrast enhancement and scale normalization to form the second image information. The feature extraction unit is used to extract key visual features from the preprocessed second image information. Based on the convolutional neural network model, it automatically learns and extracts multi-scale visual feature vectors related to agricultural threats in the image. The multi-scale visual feature vectors include, but are not limited to, the morphological texture of pests, the mottled color patches of diseased leaves, and the outline and posture of invasive birds and animals. The classification and recognition unit is used to make the final judgment of the target based on the extracted multi-scale visual feature vector. It inputs the multi-scale visual feature vector into the preset classifier for inference calculation, and then outputs the structured recognition result including the target category, location bounding box and confidence level. It compares the result with the preset model in real time to complete the recognition of specific pests or bird and animal invasion targets. The cloud computing center is used to execute global scheduling and data analysis tasks, and includes: a path planning module and a data analysis module; wherein: The path planning module generates globally optimal patrol paths for terminal devices. Based on farmland geographic information, historical patrol data, and real-time task requests, it formulates a land-air collaborative patrol plan covering the entire monitoring area. It includes: a global path unit, a task allocation unit, and a dynamic adjustment unit; wherein: The global path unit is used to generate the basic land movement path of the robot dog and the basic flight route of the drone based on map information and mission objectives using the Dijkstra algorithm. The task allocation unit is used to assign specific patrol tasks and areas to the most suitable terminal devices based on the device status and location of the robot dog, drone, and camera, so as to achieve load balancing and optimal efficiency. The dynamic adjustment unit is used to receive abnormal alarms from edge computing nodes and perform real-time global path replanning for the affected area, directing terminal tasks to respond to emergencies. The data analysis module is used for in-depth analysis and model optimization of the global data collected by the system. It receives and integrates structured identification results from edge computing nodes, status information from terminal devices, and external environmental data to make comprehensive decisions. It includes: a data fusion unit and a decision generation unit; wherein: The data fusion unit is used to perform spatiotemporal alignment and correlation of multi-source heterogeneous data to construct a unified situational view that includes image recognition results, device location, and environmental factors. The decision generation unit is used to perform trend analysis and prediction based on the global data fused by the data fusion unit, and to generate high-level decision information, including early warning of pest and disease outbreak risks, suggestions for strengthening monitoring in key areas, and the generation of periodic inspection reports.

[0023] Example 1 This embodiment 1 uses a 300-mu (approximately 20 hectares) paddy field as an application scenario to detail the operation method of the land-air collaborative agricultural patrol system based on vision and path planning of this invention for land-air collaborative agricultural patrol, including the following steps: S1. System initialization; Load pre-stored farmland geographic information maps onto the cloud computing center server, such as... Figure 3 As shown, the rasterized map resolution is 0.5m / pixel, historical patrol data (patrol routes and records for the past 30 days), a pre-set agricultural threat model (including 63 types of pests and diseases and 32 types of bird and animal invasion features), and real-time status information of all terminal devices, including robot dogs, drones, and cameras. Robot dog: Location (10,10,0), battery 95%, sensors normal; Drone: Location (50,50,30), Battery 80%, Sensors functioning normally; Camera: Position (150,100,5), fixed facing due north, 60° downward angle; S2, Global Path Planning and Task Allocation; The path planning module of the cloud computing center is based on the pre-stored farmland geographic information map and historical inspection data loaded in S1. It uses Dijkstra's algorithm through its global path unit. The specific calculation formula is as follows: The terrain complexity coefficient for this patrol is α=0.3, the historical threat coefficient is β=0.2, the equipment energy consumption coefficient is γ=0.1, the starting point is (0,0,0), and the ending point is (300,200,0). Using this data, the basic land path for the robot dog and the basic flight path for the drone covering the entire monitoring area are calculated. The shortest path total cost C_total=1246.8. A global path instruction is generated. Subsequently, the task allocation unit of the path planning module, based on the real-time status information and location of the terminal devices obtained in S1, assigns specific patrol areas and tasks to the corresponding robot dog, drone, and camera, forming a global patrol plan that coordinates land and air operations. The specific calculation formula is as follows: α²=0.4, β²=0.3, γ²=0.3, ε=0.1, Allocation results: Robot dog is responsible for the area [0,100]×[0,100] (S=0.92), drone is responsible for the area [0,300]×[100,200] (S=0.88), and camera is responsible for the fixed monitoring area [150,200]×[50,150] (S=0.95). S3. Generation and distribution of collaborative control commands; The collaborative control module of the edge computing node receives global path commands generated by the path planning module in S2 and parses the commands: The land control unit of the collaborative control module generates motion control signals for the robot dog, including linear velocity and angular velocity, based on the global path instructions and the robot dog's real-time pose. The specific calculation formula is as follows. Based on the kinematic control model, the specific calculation formula is as follows: K_p=1.2, Ki=0.1, K_d=0.05, target point (50,50), pose error e(t)=56.57, output linear velocity v=0.8 m / s, angular velocity ω=0.1 rad / s; The airborne control unit of the collaborative control module parses the global path command into UAV flight control signals containing specific flight waypoints, altitudes, and attitudes. The specific calculation formula is as follows; The specific calculation formula, combined with the heading angle wind compensation strategy, is as follows: wind speed ,wind direction Compensated heading angle ; The camera control unit of the collaborative control module generates camera control signals for adjusting the gimbal rotation angle and switching preset positions based on the collaborative strategy in the global path instructions. The specific calculation formula is as follows; The specific calculation formula, combined with field-of-view coverage optimization, is as follows: Calculate the maximum coverage setting: target point (160, 110, 0). , ; The collaborative control module on the edge computing node sends the generated motion control signals, flight control signals, and camera control signals to the corresponding robot dog, drone, and camera, respectively. S4. Farmland Environment Image Acquisition: The terminal device executes the control signal issued by S3 and synchronously acquires farmland environment images through its image acquisition module, as detailed below: The robot dog collects close-up images of the ground while moving on land; The drone acquires wide-area aerial images during flight; The camera continuously acquires images at a fixed location; the specific calculation formula is as follows. Output the first image information , t, , K, d}; The image acquisition module will send the acquired first image information, which includes raw image data and embedded metadata, to the edge computing node through its network transmitter. S5, Image Recognition and Threat Target Detection; The target recognition module of the edge computing node receives and processes the first image information from S4: The image preprocessing unit of the target recognition module performs normalization processing on the first image information, including Gaussian filtering for denoising and histogram equalization enhancement, to form the second image information. The specific calculation formula for the Gaussian filtering denoising operation is as follows: The specific calculation formula for histogram equalization enhancement is as follows: σ=1.5, k=3, L=256, M×N=1920×1080; The feature extraction unit of the target recognition module, based on a convolutional neural network model, automatically extracts multi-scale visual feature vectors related to agricultural threats from the second image information. The calculation process includes: The specific calculation formula for convolutional feature mapping is as follows: Global average pooling is calculated using the following formula: The classification and recognition unit of the target recognition module extracts multi-scale visual feature vectors. The system inputs a pre-defined classifier for inference calculations and then outputs a structured recognition result containing the target category, location bounding box, and confidence score, thus completing the identification of specific pests or invasive bird and animal targets. The calculation process includes: The specific formula for target classification and confidence score calculation is as follows: The specific formula for world coordinate system transformation is as follows: With a maximum confidence level of conf=0.12 (below the threshold of 0.7), no abnormal targets are identified. S6. Upload the recognition results; the edge computing node will upload the structured recognition results obtained from the target recognition module in S5: { "target_class": "none", "confidence": 0.12, "bbox": null, "world_coord": null} is uploaded to the data analysis module of the cloud computing center; S7, Multi-source Data Fusion and Situation Assessment: The data fusion unit of the data analysis module in the cloud computing center receives structured recognition results from S6, real-time status information from terminal devices, and external environmental data. It then uses a Kalman filter algorithm to fuse and estimate the state of the multi-source heterogeneous data from edge computing nodes, constructing a unified global situational view, such as... Figure 3 As shown, the fusion process is as follows: Spatiotemporal alignment and fusion processing are performed on these multi-source heterogeneous data, with state vector x=[pos_x, pos_y,pos_z, v_x, v_y, v_z] and observation vector z=[GPS, IMU, image_coord]; S8, Data Analysis and Decision Generation; the decision generation unit of the data analysis module in the cloud computing center, which performs in-depth analysis based on the fused global situational view data in S7: The specific formula for calculating the agricultural threat risk score is as follows; The feature vector X = [0.1, 0.05, 0.02, ...], the weight vector Λ = [0.15, 0.12, 0.08, ...], and the output RiskScore = 0.084, which is qualitatively classified as low risk. An autoregressive integral moving average model was used to predict the time series data of pests and diseases. The results showed p=2, d=1, q=1, predicting no risk of pest or disease outbreaks in the next 7 days. Modeling the spatial spread trend of pests and diseases based on reaction-diffusion equations; D=0.01, α=0.1, K=100, β=0.05, Output result: No significant propagation trend. Based on the calculation and analysis results, the following decision information is generated: No warning is given, and it is recommended to keep the default monitoring. S9, Anomaly Response and Dynamic Path Adjustment; The dynamic adjustment unit of the path planning module in the cloud computing center continuously monitors the decision information generated by the data analysis module in S8 and the recognition results uploaded in S6; when it receives an anomaly alarm generated by the target recognition module in S5, it triggers a dynamic response strategy: W_a=0.3, W_b=0.8, W=0.5, lab="none", execute the default strategy, no path replanning; S10, Collaborative Control Closed-Loop Execution: The collaborative control module of the edge computing node receives the new path instructions generated after dynamic adjustment in S9, and repeats the process of S3 to generate new control signals and send them to the corresponding terminal devices to instruct them to perform drive-away tasks or strengthen monitoring of key areas, thus forming closed-loop control. S11. Continuous system operation and optimization: The system repeats steps S4 to S10 to achieve 24-hour uninterrupted land-air coordinated agricultural patrol.

[0024] Example 2 This embodiment 2 uses a 300-mu (approximately 20 hectares) paddy field as an application scenario to detail the operation method of the land-air collaborative agricultural patrol system based on vision and path planning of the present invention for land-air collaborative agricultural patrol, including the following steps: Step 1: System initialization; The cloud computing center loads a pre-stored farmland geographic information map, such as... Figure 3 As shown, the rasterized map resolution is 0.5m / pixel, historical patrol data (patrol routes and records for the past 30 days), a pre-set agricultural threat model (including 63 types of pests and diseases and 32 types of bird and animal invasion features), and real-time status information of all registered terminal devices, including robot dogs, drones, and cameras. Robot dog: Location (10,10,0), battery 99%, sensors normal; Drone: Location (50,50,30), Battery 98%, Sensors functioning normally; Camera: Position (150,100,5), fixed facing due north, 60° downward angle; Step 2: Global Path Planning and Task Allocation; The path planning module of the cloud computing center, based on the farmland geographic information map and historical inspection data loaded in Step 1, uses Dijkstra's algorithm through its global path units. The specific calculation formula is as follows: The terrain complexity coefficient for this patrol is α=0.3, the historical threat coefficient is β=0.2, the equipment energy consumption coefficient is γ=0.1, the starting point is (0,0,0), and the ending point is (300,200,0). Substituting this data, the total cost of the shortest path for the robot dog's basic land movement and the drone's basic air flight path, covering the entire monitoring area, is calculated to be C_total=1246.8. Subsequently, the task allocation unit of the path planning module, based on the real-time status information and location of the terminal devices obtained in step 1, assigns specific patrol areas and tasks to the corresponding robot dogs, drones, and cameras, forming a global patrol plan that coordinates land and air operations. The specific calculation formula is as follows: α²=0.4, β²=0.3, γ²=0.3, ε=0.1, Allocation results: Robot dog is responsible for the area [0,100]×[0,100] (S=0.92), drone is responsible for the area [0,300]×[100,200] (S=0.88), and camera is responsible for the fixed monitoring area [150,200]×[50,150] (S=0.95). Step 3: Generation and distribution of collaborative control instructions; The collaborative control module of the edge computing node receives the global path instruction generated by the path planning module in Step 2 and parses the instruction: The land control unit of the collaborative control module generates motion control signals for the robot dog, including linear velocity and angular velocity, based on the global path instructions and the robot dog's real-time pose. The specific calculation formula is as follows. Based on the kinematic control model, the specific calculation formula is as follows: K_p=1.2, Ki=0.1, K_d=0.05, target point (50,50), pose error e(t)=56.57, output linear velocity v=0.8 m / s, angular velocity ω=0.1 rad / s; The airborne control unit of the collaborative control module parses the global path command into UAV flight control signals containing specific flight waypoints, altitudes, and attitudes. The specific calculation formula is as follows; The specific calculation formula, combined with the heading angle wind compensation strategy, is as follows: wind speed ,wind direction Compensated heading angle ; The camera control unit of the collaborative control module generates camera control signals for adjusting the gimbal rotation angle and switching preset positions based on the collaborative strategy in the global path instructions. The specific calculation formula is as follows; The specific calculation formula, combined with field-of-view coverage optimization, is as follows: Calculate the maximum coverage setting: target point (160, 110, 0). , ; The collaborative control module on the edge computing node sends the generated motion control signals, flight control signals, and camera control signals to the corresponding robot dog, drone, and camera, respectively. Step 4: Farmland Environment Image Acquisition; The terminal device executes the control signal issued in Step 3 and synchronously acquires farmland environment images through its image acquisition module, as detailed below: The robot dog collects close-up images of the ground while moving on land; The drone acquires wide-area aerial images during flight; The camera continuously acquires images at a fixed location; the specific calculation formula is as follows. Output the first image information ; The image acquisition module will send the acquired first image information, which includes raw image data and embedded metadata, to the edge computing node through its network transmitter. Step 5, Image Recognition and Threat Target Detection; The target recognition module of the edge computing node receives and processes the first image information from Step 4: The image preprocessing unit of the target recognition module performs normalization processing on the first image information, including Gaussian filtering for denoising and histogram equalization enhancement, to form the second image information. The specific calculation formula for the Gaussian filtering denoising operation is as follows: The specific calculation formula for histogram equalization enhancement is as follows: σ=1.5, k=3, L=256, M×N=1920×1080; The feature extraction unit of the target recognition module, based on a convolutional neural network model, automatically extracts multi-scale visual feature vectors related to agricultural threats from the second image information. The calculation process includes: The specific calculation formula for convolutional feature mapping is as follows: Global average pooling is calculated using the following formula: The classification and recognition unit of the target recognition module extracts multi-scale visual feature vectors. The system inputs a pre-defined classifier for inference calculations and then outputs a structured recognition result containing the target category, location bounding box, and confidence score, thus completing the identification of specific pests or invasive bird and animal targets. The calculation process includes: The specific formula for target classification and confidence score calculation is as follows: The specific formula for world coordinate system transformation is as follows: In position The pest "rice planthopper" was detected at [location name], with a confidence level of conf=0.85 (above the threshold of 0.7), bounding box dimensions h=12, w=8, and world coordinates [value]. ; Step 6: Upload the recognition results; the edge computing node will upload the structured recognition results from the target recognition module obtained in Step 5: { "target_class": "rice planthopper", "confidence": 0.85, "bbox": , "world_coord": The data is uploaded to the data analysis module in the cloud computing center. Step 7: Multi-source data fusion and situational awareness construction; The data fusion unit of the data analysis module in the cloud computing center receives the structured recognition results from Step 6, real-time status information from terminal devices, and external environmental data. It then uses the Kalman filter algorithm to fuse and estimate the state of the multi-source heterogeneous data from edge computing nodes, constructing a unified global situational awareness view, such as... Figure 3 As shown, the fusion process is as follows: Spatiotemporal alignment and fusion processing are performed on these multi-source heterogeneous data, with state vector x=[pos_x, pos_y,pos_z, v_x, v_y, v_z] and observation vector z=[GPS, IMU, image_coord]; Step 8: Data Analysis and Decision Generation; The decision generation unit of the data analysis module in the cloud computing center performs in-depth analysis based on the fused global situational view data from Step 7: The specific formula for calculating the agricultural threat risk score is as follows; Feature vector X = [0.8, 0.65, 0.44, ...], weight vector Λ = [0.25, 0.2, 0.58, ...], output RiskScore = 0.421, qualitatively classified as medium risk; An autoregressive integral moving average model was used to predict the time series data of pests and diseases. The results showed p=4, d=2, q=2, predicting a moderate outbreak risk of rice planthoppers in the region over the next 7 days. Modeling the spatial spread trend of pests and diseases based on reaction-diffusion equations; With D=0.02, α=0.18, K=150, β=0.1, the output shows that the pests and diseases will spread to the surrounding area at a rate of 15m / day. Based on the calculation and analysis results, the following decision information is generated: a moderate risk warning for pests and diseases. It is recommended to strengthen monitoring within a 50m radius of coordinates (84.2, 74.5) and increase the patrol frequency to 3 times per day. Step 9, Anomaly Response and Dynamic Path Adjustment; The dynamic adjustment unit of the path planning module in the cloud computing center continuously monitors the decision information generated by the data analysis module in Step 8 and the recognition results uploaded in Step 6; when it receives an anomaly alarm generated by the target recognition module in Step 5, it triggers a dynamic response strategy: W_a=0.3, W_b=0.8, W=0.5, lab=" The system dynamically adjusts the recording strategy of the unit, recording the coordinates (84.2, 74.5, 0.3), time (2025-07-15T14:30:00), and species "rice planthopper" of the pest occurrence to the decision generation unit in the cloud computing center. It does not trigger the expulsion task, and all devices maintain the default execution state for inspection. Step 10: Collaborative control closed-loop execution; The collaborative control module of the edge computing node receives the new path instruction generated after dynamic adjustment in step 9, and repeats the process of step 3 to generate new control signals and send them to the corresponding terminal devices, instructing them to execute the expulsion task or strengthen monitoring of key areas, forming a closed-loop control; In this example, due to the execution of the recording strategy, there is no new path instruction, and the system maintains the original patrol plan. Step 11: Continuous system operation and optimization; The system repeats steps 4 to 10 to achieve 24-hour uninterrupted land-air coordinated agricultural patrols and continuously monitor the development trend of marked pest and disease areas. Example 3 This embodiment 2 uses a 300-mu (approximately 20 hectares) paddy field as an application scenario to detail the operation method of the land-air collaborative agricultural patrol system based on vision and path planning of the present invention for land-air collaborative agricultural patrol, including the following steps: Step 1: System initialization; The cloud computing center loads a pre-stored farmland geographic information map, such as... Figure 3 As shown, the rasterized map resolution is 0.5m / pixel, historical patrol data (patrol routes and records for the past 30 days), a pre-set agricultural threat model (including 63 types of pests and diseases and 32 types of bird and animal invasion features), and real-time status information of all registered terminal devices, including robot dogs, drones, and cameras. Robot dog: Location (10,10,0), battery 95%, sensors normal; Drone: Location (50,50,30), Battery 85%, Sensors normal; Camera: Position (150,100,5), fixed facing due north, 60° downward angle; Step 2: Global Path Planning and Task Allocation; The path planning module of the cloud computing center, based on the farmland geographic information map and historical inspection data loaded in Step 1, uses Dijkstra's algorithm through its global path units. The specific calculation formula is as follows: The terrain complexity coefficient for this patrol is α=0.3, the historical threat coefficient is β=0.2, the equipment energy consumption coefficient is γ=0.1, the starting point is (0,0,0), and the ending point is (300,200,0). Substituting this data, the total cost of the shortest path for the robot dog's basic land movement and the drone's basic air flight path, covering the entire monitoring area, is calculated to be C_total=1246.8. Subsequently, the task allocation unit of the path planning module, based on the real-time status information and location of the terminal devices obtained in step 1, assigns specific patrol areas and tasks to the corresponding robot dogs, drones, and cameras, forming a global patrol plan that coordinates land and air operations. The specific calculation formula is as follows: α²=0.4, β²=0.3, γ²=0.3, ε=0.1, Allocation results: Robot dog is responsible for the area [0,100]×[0,100] (S=0.92), drone is responsible for the area [0,300]×[100,200] (S=0.88), and camera is responsible for the fixed monitoring area [150,200]×[50,150] (S=0.95). Step 3: Generation and distribution of collaborative control instructions; The collaborative control module of the edge computing node receives the global path instruction generated by the path planning module in Step 2 and parses the instruction: The land control unit of the collaborative control module generates motion control signals for the robot dog, including linear velocity and angular velocity, based on the global path instructions and the robot dog's real-time pose. The specific calculation formula is as follows. Based on the kinematic control model, the specific calculation formula is as follows: K_p=1.2, Ki=0.1, K_d=0.05, target point (50,50), pose error e(t)=56.57, output linear velocity v=0.8 m / s, angular velocity ω=0.1 rad / s; The airborne control unit of the collaborative control module parses the global path command into UAV flight control signals containing specific flight waypoints, altitudes, and attitudes. The specific calculation formula is as follows; The specific calculation formula, combined with the heading angle wind compensation strategy, is as follows: wind speed ,wind direction Compensated heading angle ; The camera control unit of the collaborative control module generates camera control signals for adjusting the gimbal rotation angle and switching preset positions based on the collaborative strategy in the global path instructions. The specific calculation formula is as follows; The specific calculation formula, combined with field-of-view coverage optimization, is as follows: Calculate the maximum coverage setting: target point (160, 110, 0). =18.4°, =56.3°; The collaborative control module on the edge computing node sends the generated motion control signals, flight control signals, and camera control signals to the corresponding robot dog, drone, and camera, respectively. Step 4: Farmland Environment Image Acquisition; The terminal device executes the control signal issued in Step 3 and synchronously acquires farmland environment images through its image acquisition module: The robot dog collects close-up images of the ground while moving on land; The drone acquires wide-area aerial images during flight; The camera continuously acquires images at a fixed location; the specific calculation formula is as follows. Output the first image information ; The image acquisition module will send the acquired first image information, which includes raw image data and embedded metadata, to the edge computing node through its network transmitter. Step 5, Image Recognition and Threat Target Detection; The target recognition module of the edge computing node receives and processes the first image information from Step 4: The image preprocessing unit of the target recognition module performs normalization processing on the first image information, including Gaussian filtering for denoising and histogram equalization enhancement, to form the second image information. The specific calculation formula for the Gaussian filtering denoising operation is as follows: The specific calculation formula for histogram equalization enhancement is as follows: σ=1.5, k=3, L=256, M×N=1920×1080; The feature extraction unit of the target recognition module, based on a convolutional neural network model, automatically extracts multi-scale visual feature vectors related to agricultural threats from the second image information; its calculation process includes: The specific calculation formula for convolutional feature mapping is as follows: Global average pooling is calculated using the following formula: The classification and recognition unit of the target recognition module extracts multi-scale visual feature vectors. The system inputs a pre-defined classifier for inference calculations and then outputs a structured recognition result containing the target category, location bounding box, and confidence score, thus completing the identification of specific pests or invasive bird and animal targets. The calculation process includes: The specific formula for target classification and confidence score calculation is as follows: The specific formula for world coordinate system transformation is as follows: In position Birds and animals were detected invading a flock of sparrows. Confidence level conf=0.91 (above the threshold of 0.7). Bounding box dimensions h=25, w=32. World coordinates. ; Step 6: Upload the recognition results; the edge computing node will upload the structured recognition results from the target recognition module obtained in Step 5: { "target_class": "sparrow flock", "confidence": 0.91, "bbox": , "world_coord": The data is uploaded to the data analysis module in the cloud computing center. Step 7: Multi-source data fusion and situational awareness construction; The data fusion unit of the data analysis module in the cloud computing center receives the structured recognition results from Step 6, real-time status information from terminal devices, and external environmental data. It then uses the Kalman filter algorithm to fuse and estimate the state of the multi-source heterogeneous data from edge computing nodes, constructing a unified global situational awareness view, such as... Figure 3 As shown, the fusion process is as follows: Spatiotemporal alignment and fusion processing are performed on these multi-source heterogeneous data, with state vector x=[pos_x, pos_y,pos_z, v_x, v_y, v_z] and observation vector z=[GPS, IMU, image_coord]; Step 8: Data Analysis and Decision Generation; The decision generation unit of the data analysis module in the cloud computing center performs in-depth analysis based on the fused global situational view data from Step 7: The specific formula for calculating the agricultural threat risk score is as follows; The feature vector X = [0.9, 0.75, 0.6, ...], the weight vector Λ = [0.3, 0.2, 0.15, ...], and the output RiskScore = 0.523, which is qualitatively classified as high risk. An autoregressive integral moving average model was used to predict the time series data of pests and diseases. The results showed p=3, d=1, q=3, predicting that the bird flock's activity would last for 2-3 hours. Modeling the spatial propagation trend of harmful birds based on the reaction-diffusion equation; With D=0.02, α=0.15, K=229, β=0.1, the output results show that the bird flock is mobile and may migrate southeast. Based on the calculation and analysis results, the following decision information is generated: High risk warning for bird and animal invasion, it is recommended to immediately carry out the expulsion task and track its movement trajectory; High risk warning for harmful birds, it is recommended to strengthen the monitoring of the 150m range around coordinates (219, 129) and increase the patrol frequency to twice a day. Step 9, Anomaly Response and Dynamic Path Adjustment; The dynamic adjustment unit of the path planning module in the cloud computing center continuously monitors the decision information generated by the data analysis module in Step 8 and the recognition results uploaded in Step 6; when it receives an anomaly alarm generated by the target recognition module in Step 5, it triggers a dynamic response strategy: W_a=0.6, W_b=0.2, W=0.8, lab=" The dynamic adjustment unit executes the expulsion strategy: First, it records the intrusion coordinates (219.5, 129.8, 1.2), and then immediately sends a high-priority expulsion task to the task allocation unit. The task allocation unit calculates, based on the decision function S(d,t), that the closest and fastest-responding terminal device in the current state is the drone (current position (50,50,30) to the target point distance is 175.3m, S=0.91, higher than the robot dog's S=0.78), and assigns it to that coordinate. The global path unit then plans an optimal path for the drone to the intrusion coordinates, with the initial path point... =[(50,50,30), (125,90,25), (219.5,129.8,15)], and optimize the path in real time using gradient descent, where the learning rate η=0.01, and after one iteration... =[(50,50,30), (120,85,24), (219.5,129.8,15)], the total path cost C_total is reduced by 12.6%; Step 10: Collaborative Control Closed-Loop Execution; The collaborative control module of the edge computing node receives the new path instruction generated after dynamic adjustment in Step 9, and repeats the process of Step 3, generating new control signals and sending them to the corresponding terminal devices to instruct them to execute the expulsion task. Specifically, the airborne control unit generates a flight control signal to accelerate the assigned UAV, increasing its speed to 3.5 m / s, while adjusting its heading angle to directly point to the intrusion coordinates; the land control unit adjusts the robot dog's patrol path to enable it to collaboratively monitor the surrounding area; the camera control unit turns the gimbal to the expulsion area for tracking and shooting, forming a closed-loop control. Step 11: System continuous operation and optimization; The system repeats steps 4 to 10 to continuously track the movement trajectory of bird and animal flocks, and automatically restores the default patrol mode after successful dispersal, realizing 24-hour uninterrupted land-air coordinated agricultural patrol.

[0025] The content disclosed above is only a preferred and feasible embodiment of the present invention, and is not intended to limit the scope of protection of the present invention. Therefore, all equivalent technical changes made based on the content of the present invention specification and drawings are included within the scope of protection of the present invention. Furthermore, the elements therein can be updated as technology develops.

Claims

1. A land-air collaborative agricultural patrol system based on vision and path planning, comprising a path planning module and a data analysis module running in a cloud computing center, a collaborative control module and a target recognition module running on edge computing nodes, and an image acquisition module running on a terminal device; characterized in that: The terminal equipment includes a robot dog, a drone, and a camera, wherein: The image acquisition module on the robot dog is used to acquire images of farmland from close range on the ground and for local monitoring; The image acquisition module on the drone is used to acquire wide-area images of farmland from the air and to conduct large-scale inspections; The image acquisition module on the camera is used to continuously acquire images of farmland from fixed locations; The image acquisition module is connected to the edge computing node and includes: an optical image sensing camera, a buffer register, and a network transmitter; wherein: An optical image sensing camera is used to capture images of farmland and its surrounding environment, and to convert the light signals of the images of farmland and its surrounding environment into first image information. The buffer register is used to store the first image information; The network transmitter is used to transmit the first image information to the edge computing node; The edge computing node is connected to both the terminal device and the cloud computing center, and is used to receive and process first image information from the terminal device and data from the cloud computing center. It includes: a collaborative control module and a target recognition module, wherein: The collaborative control module receives and parses global path instructions from the cloud computing center, generates specific action control signals based on the real-time context, and distributes them to the corresponding robot dog, drone, and camera on the terminal devices. It includes: a land control unit, an air control unit, and a camera control unit, wherein: The land control unit is used to control the robot dog's behavior. It performs dynamic path planning and local obstacle avoidance based on global path instructions in order to reach the designated location for inspection. The airborne control unit is used to control the behavior of the drone. It parses the global path instructions into specific flight waypoints and flight altitudes, and directs the drone to complete fixed-point area scanning and inspection of a large area of ​​farmland. The camera control unit is used to control cameras at fixed locations. Based on global commands and collaborative strategies, it generates control signals, including pan-tilt rotation angle and preset position switching, to adjust the monitoring field of view. The target recognition module analyzes and recognizes the first image information uploaded by the image acquisition module, extracts feature information from the image, and compares it with a pre-set agricultural threat model. It includes: an image preprocessing unit, a feature extraction unit, and a classification and recognition unit, wherein: The image preprocessing unit is used to standardize the first image information, and its operations include image denoising, contrast enhancement and scale normalization to form the second image information. The feature extraction unit is used to extract key visual features from the preprocessed second image information. Based on the convolutional neural network model, it automatically learns and extracts multi-scale visual feature vectors related to agricultural threats in the image. The multi-scale visual feature vectors include, but are not limited to, the morphological texture of pests, the mottled color patches of diseased leaves, and the outline and posture of invasive birds and animals. The classification and recognition unit is used to complete the final determination of the target based on the extracted multi-scale visual feature vectors. It inputs the multi-scale visual feature vectors into a pre-set classifier for inference calculation, and then outputs a structured recognition result including the target category, position bounding box, and confidence level, and compares it with the pre-set model in real time to complete the recognition of specific pest and disease species or bird and animal invasion targets; The cloud computing center is used to perform global scheduling and data analysis, and it includes: a path planning module, a data analysis module; where: The path planning module is used to generate the globally optimal inspection path for the terminal device. Based on the farmland geographic information, historical inspection data, and real-time task requests, it formulates a land-air collaborative inspection plan covering all monitoring areas, and it includes: a global path unit, a task allocation unit, a dynamic adjustment unit; where: The global path unit is used to generate the basic land travel path of the robot dog and the basic air flight route of the drone based on the map information and task objectives using the Dijkstra algorithm; The task allocation unit is used to allocate specific inspection tasks and areas to the most suitable terminal device according to the device status and location of the robot dog, drone, and camera to achieve load balancing and optimal efficiency; The dynamic adjustment unit is used to receive the abnormal alarm from the edge computing node and perform real-time re-planning of the global path for the affected area, and direct the terminal task to respond to emergencies; The data analysis module is used to perform in-depth analysis and model optimization on the global data collected by the system. It receives and fuses the structured recognition results from the edge computing node, the status information of the terminal device, and the external environment data, and makes comprehensive decisions. It includes: a data fusion unit, a decision generation unit; where: The data fusion unit is used to perform spatio-temporal alignment and association on multi-source heterogeneous data, and construct a unified situation view including image recognition results, device locations, and environmental factors; The decision generation unit is used to perform trend analysis and prediction based on the global data fused by the data fusion unit, and generate high-level decision-making information including pest and disease outbreak risk warnings, suggestions for strengthening monitoring in key areas, and generating periodic inspection reports.

2. The land-air collaborative agricultural patrol system based on vision and path planning according to claim 1, characterized in that, The global path unit of the path planning module in the cloud computing center first executes the default patrol route. This default patrol route is a default patrol route covering the entire domain, calculated using Dijkstra's algorithm. The total path cost C_total is calculated based on weight coefficients. Euclidean distance The specific calculation formula is as follows: In the formula, Indicates from coordinate point to coordinate point The weighting coefficients are: α is the terrain complexity coefficient, β is the historical threat coefficient, and γ is the equipment energy consumption coefficient. Represents the coordinates of the node. to coordinate point The Euclidean distance, Σ, represents the sum of the costs of all edges along the entire path p. The cost is the shortest path from the starting point to node j. The task allocation unit of the path planning module of the cloud computing center assigns appropriate terminal devices to each newly generated inspection task based on the following decision function S(d,t): In the formula, S(d,t) represents the fitness score of device d for task t. E(d) represents the Euclidean distance from the current coordinate point d to the task coordinate point t, and E(d) represents the remaining battery power of device d. α2, β2, γ2 are the normalized weight coefficients of each factor; the task allocation unit solves the above optimization problem and allocates the task to the terminal device that maximizes the total adaptability score. The dynamic adjustment unit of the path planning module of the cloud computing center continuously listens to the target information received by the data analysis module during the operation period. When it receives a pest and disease recognition alarm or a bird and animal invasion alarm generated by the target recognition module on the edge computing node, it will load the predefined response weights Wa, Wb and compare them with the default weight W, and trigger a differentiated response strategy: The dynamic adjustment unit of the path planning module of the cloud computing center triggers a differentiated response strategy according to the value of the default weight W: If the recognized target is a pest and disease, when Wa < W, the dynamic adjustment unit executes the recording strategy, that is, records the coordinates, time, and species information of the occurrence of the pest and disease into the decision generation unit of the cloud computing center; If the target is identified as an intrusion by birds or animals, the Wb>W dynamic adjustment unit executes a deportation strategy. First, it records the intrusion coordinates and then immediately sends a high-priority deportation task to the task allocation unit. The task allocation unit calculates the nearest and fastest-responding terminal device in the current state based on the weighted decision function S(d,t) and assigns it to that coordinate. Finally, the global path unit plans an optimal path to the intrusion coordinates for the assigned device and optimizes the path in real time using gradient descent. Let η represent the path point in the k-th iteration, and η be the learning rate.

3. The land-air collaborative agricultural patrol system based on vision and path planning according to claim 1, characterized in that, The data fusion unit of the data analysis module in the cloud computing center uses the Kalman filter algorithm to fuse and estimate the state of multi-source heterogeneous data from edge computing nodes, constructing a unified global situational view. The fusion process is as follows: in, This is the predicted state value. State transition matrix, To predict covariance, For process noise covariance, For Kalman gain, For the observation vector, For the observation matrix, To observe the noise covariance, and These are the updated state estimates and covariance, respectively. The decision generation unit of the data analysis module in the cloud computing center performs risk scoring, trend prediction, and spatial propagation modeling based on the fused global data, and generates decision information including early warnings, key monitoring recommendations, and inspection reports based on the output results. Specifically, it includes the following steps: (1) Calculate the agricultural threat risk score based on the multi-dimensional feature vector X and the weight vector Λ: (2) The autoregressive integral moving average model was used to model and predict the time series data of pests and diseases: in, For lag operators, These are time series observations. For model parameters, It is white noise; (3) Modeling the spatial spread trend of pests and diseases based on the reaction-diffusion equation: in, For pest and disease density, Let α be the diffusion coefficient and α be the natural growth rate. β represents the environmental capacity, and β represents the control intensity. Based on the output of the above model, the decision generation unit automatically generates decision recommendations including risk level, predicted trend, and propagation path, and triggers corresponding early warning or control instructions.

4. The land-air collaborative agricultural patrol system based on vision and path planning according to claim 1, characterized in that, The image preprocessing unit of the target recognition module on the edge computing node performs standardization processing on the first image information uploaded by the image acquisition module on the terminal device to form second image information. The processing procedure includes: Gaussian filtering noise reduction operation: in, This is the first image information. For Gaussian kernel function, Standard deviation; Histogram equalization enhancement operation: in, For gray levels, grayscale The number of pixels, Image size; The feature extraction unit of the target recognition module on the edge computing node extracts multi-scale visual features from the preprocessed second image information based on a convolutional neural network. Its calculation process includes: Convolutional feature mapping: in, For the feature map of layer l, For convolution kernel weights, For bias terms, It is the ReLU activation function; Global average pooling: in, For the final feature vector, For feature dimensions; The classification and recognition unit of the target recognition module on the edge computing node completes the final recognition and localization of the target based on the extracted feature vectors. Its calculation process includes: Target classification and confidence calculation: in, Let be the posterior probability of class c, be the predicted class label, and be the confidence level. Target bounding box regression: in, The coordinates of the target center are This refers to the bounding box dimensions; World coordinate system transformation: in, For the camera intrinsic parameter matrix, For depth information, This represents the camera's position in the world coordinate system.

5. The land-air collaborative agricultural patrol system based on vision and path planning according to claim 1, characterized in that, The land control unit of the collaborative control module of the edge computing node performs motion control on the robot dog of the terminal device based on the global path information generated by the path planning module of the cloud computing center. Its calculation process includes: Path cost control: The land control unit of the collaborative control module of the edge computing node receives global path instructions and path cost information, including the minimum path cost, from the path planning module of the cloud computing center. It is used to guide the robot dog's local movement decisions; PID motion control: in, This represents the pose error between the robot dog's current position and the target position in the global path command. These are the proportional, integral, and differential coefficients, respectively. Kinematic control model: in, These are the robot dog's linear velocity and angular velocity, respectively, which are adjusted in real time based on the path point sequence output by the path planning module. The airborne control unit of the collaborative control module of the edge computing node performs flight control based on the UAV dynamics model and wind resistance compensation strategy of the terminal device. Its calculation process includes: Dynamic equations: in, For the quality of drones, For position vectors, Let be a rotation matrix. The lift vector, Let ω be the moment of inertia, ω be the angular velocity, and τ be the torque. Quaternion pose representation: Heading angle wind compensation: in, The compensated heading angle, For wind speed, Wind direction; The camera control unit of the collaborative control module of the edge computing node generates control signals based on a preset positioning strategy and a field-of-view optimization model, including: Preset position switching control: Based on global commands and collaborative strategies, the system automatically switches to the preset monitoring position and adjusts the monitoring field of view to cover key areas. The preset position is due north with an elevation angle of 60 degrees. Gimbal rotation control: Based on the target location information, the pan-tilt-zoom (PTZ) rotation angle is calculated to ensure the target remains centered in the monitoring field of view. The formula for generating the control signal is as follows: In the formula Using the target world coordinates, Location for camera installation; Optimized field of view coverage: in, For a moment Surveillance coverage Assuming energy consumption costs, α and β are weighting coefficients. To maximize monitoring coverage, the collaborative monitoring strategy of multiple cameras is dynamically adjusted.

6. The land-air collaborative agricultural patrol system based on vision and path planning according to claim 1, characterized in that, The optical image sensing camera of the image acquisition module of the terminal device is used to acquire raw image data of the farmland environment, and generates first image information for use by the target recognition module through the following processing: Image acquisition and digital processing are performed according to the following formula: in, The raw light signal collected by the sensor. The output digital image after discrete cosine transform. These are the transformation coefficients; The optical image sensing camera also outputs embedded metadata, including: acquisition timestamp t, device location coordinates. Camera intrinsic parameter matrix K, depth information d; Embedded metadata and Together they constitute the first image information .

7. An operational method for conducting land-air coordinated agricultural patrols using a vision and path planning-based land-air coordinated agricultural patrol system, characterized in that... Includes the following steps: S1: System initialization; The cloud computing center loads pre-stored farmland geographic information maps, historical inspection data, and pre-set agricultural threat models, and obtains real-time status information of all terminal devices through edge computing nodes, including robot dogs, drones, and cameras; S2: Global Path Planning and Task Allocation; The path planning module of the cloud computing center, based on the pre-stored farmland geographic information map and historical patrol data loaded in S1, uses the Dijkstra algorithm through its global path unit to calculate the basic land travel path of the robot dog and the basic flight route of the drone covering the entire monitoring area, and generates global path instructions; Then, the task allocation unit of the path planning module of the cloud computing center, according to the real-time status information and location of the terminal devices obtained in S1, assigns the specific patrol area and tasks to the corresponding robot dog, drone and camera, forming a global patrol plan that coordinates land and air. S3: Cooperative control command generation and distribution; the cooperative control module of the edge computing node receives the global path command generated by the path planning module in S2 and parses the command: The land control unit of the collaborative control module generates motion control signals for the robot dog, including linear velocity and angular velocity, based on the global path instructions and the robot dog's real-time pose. The airborne control unit of the collaborative control module parses the global path command into UAV flight control signals containing specific flight waypoints, altitudes, and attitudes; The camera control unit of the collaborative control module generates camera control signals for adjusting the gimbal rotation angle and switching preset positions according to the collaborative strategy in the global path instructions; the collaborative control module on the edge computing node sends the generated motion control signals, flight control signals and camera control signals to the corresponding robot dog, drone and camera respectively; S4: Farmland environment image acquisition; The terminal device executes the control signal sent by S3 and synchronously acquires farmland environment images through its image acquisition module: The robot dog collects close-up images of the ground while moving on land; The drone acquires wide-area aerial images during flight; The camera continuously captures images from a fixed location; The image acquisition module will send the acquired first image information, which includes raw image data and embedded metadata, to the edge computing node through its network transmitter. S5: Image recognition and threat target detection; the target recognition module of the edge computing node, which receives and processes the first image information from S4. The image preprocessing unit of the target recognition module performs normalization processing on the first image information, including Gaussian filtering for noise reduction and histogram equalization enhancement, to form the second image information; The feature extraction unit of the target recognition module automatically extracts multi-scale visual feature vectors related to agricultural threats from the second image information based on a convolutional neural network model; The classification and recognition unit of the target recognition module inputs the extracted multi-scale visual feature vectors into a preset classifier for inference calculation, and then outputs a structured recognition result containing the target category, location bounding box and confidence level, thus completing the recognition of specific pests or invasive bird and animal targets. S6: Upload the recognition results; The edge computing node uploads the structured recognition results obtained from the target recognition module in S5 to the data analysis module of the cloud computing center; S7: Multi-source data fusion and situational awareness construction; The data fusion unit of the data analysis module in the cloud computing center receives structured recognition results from S6, real-time status information from terminal devices, and external environmental data. It then uses the Kalman filter algorithm to perform spatiotemporal alignment and fusion processing on these multi-source heterogeneous data to construct a unified global situational awareness view. S8: Data Analysis and Decision Generation; the decision generation unit of the data analysis module in the cloud computing center, performing the following in-depth analysis based on the fused global situational view data from S7: Calculate agricultural threat risk scores; An autoregressive integral moving average model was used to predict the time series data of pests and diseases. Based on the reaction-diffusion equation, the spatial spread trend of pests and diseases is modeled; Based on the above analysis results, high-level decision-making information is generated, including early warning of pest and disease outbreak risks, recommendations for strengthening monitoring in key areas, and periodic inspection reports; S9: Anomaly Response and Dynamic Path Adjustment; The dynamic adjustment unit of the path planning module in the cloud computing center continuously monitors the decision information generated by the data analysis module in S8 and the recognition results uploaded in S6; when an anomaly alarm generated by the target recognition module in S5 is received, a dynamic response strategy is triggered: If the identified target is a pest or disease, the dynamic adjustment unit executes the recording strategy and records the location, time, and type information of the event to the decision generation unit in S8. If the target is identified as an intrusion by birds or animals, the dynamic adjustment unit executes a deportation strategy: first, it records the intrusion coordinates, and then immediately sends a high-priority deportation task to the task allocation unit; the task allocation unit allocates the device with the closest location and fastest response from the terminal devices to execute the deportation task based on the decision function S(d,t); the global path unit then plans an optimal path to the intrusion coordinates for the device. S10: Collaborative control closed-loop execution; The collaborative control module of the edge computing node receives the new path instructions generated after dynamic adjustment in S9, and repeats the process of S3 to generate new control signals and send them to the corresponding terminal devices, instructing the terminal devices to perform drive-away tasks or to strengthen monitoring of key areas, thus forming closed-loop control. S11: System continuous operation and optimization; the system repeats steps S4 to S10 to achieve 24-hour uninterrupted land-air coordinated agricultural patrol.

Images