An active interactive pest inspection intelligent electronic eye system and method

Abstract

The present disclosure provides an active interactive pest inspection intelligent electronic eye system and method. The system is installed on a mobile platform, which includes a mobile platform chassis (1), a gimbal mast (2), a depth camera (3), an infrared camera (4), an active near-infrared lighting ring (5), a mechanical arm (6), a flexible actuator (7), a computing unit (8), a body (9), a communication and positioning unit (10), and an anti-glare shade (11). The body (9) is arranged on the mobile platform chassis (1), and the body (9) is provided with a power supply. The mechanical arm (6) and the computing unit (8) are respectively arranged at both ends of the upper surface of the body (9), the first end of the mechanical arm (6) is connected to the flexible actuator (7), and the second end of the mechanical arm (6) close to the flexible actuator (7) is provided with the gimbal mast (2). The communication and positioning unit (10) is arranged on the computing unit (8). The anti-glare shade (11), the active near-infrared lighting ring (5), the depth camera (3), and the infrared camera (4) are all arranged on the gimbal mast (2).

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of intelligent sensing and machine vision, in particular to an active interactive plant disease and pest inspection intelligent electronic eye system and method. BACKGROUND

[0002] The existing plant disease and pest monitoring and inspection technology mainly relies on visible light and hyperspectral imaging. Firstly, the visible light scheme is sensitive to natural light changes, imaging angles, leaf surface specular highlights and canopy shielding, and it is difficult to obtain key clues from the back of the leaf and the shielding part, resulting in unstable identification of subtle signs such as leaf back egg, early disease spot and leaf tunnel. Secondly, although the hyperspectral scheme has the advantage of spectral discrimination, the cost of equipment and algorithm chain is high, the volume and energy consumption are large, and the data processing overhead is heavy, which is not conducive to the implementation of motorization, large-scale deployment and long-term maintenance in the field of plant quarantine and rapid screening of invasive pests in greenhouses, orchards, forests, urban green belts, and ports / storage / nurseries. At the same time, the existing methods are mostly passive observation, and lack the ability to actively obtain canopy shielding and leaf back information, making it difficult to form a deterministic observation of the target part. SUMMARY

[0003] (I) Technical problems to be solved

[0004] In view of the above problems, the present disclosure provides an active interactive plant disease and pest inspection intelligent electronic eye system and method to at least partially solve the problems of traditional passive observation, systematic missed detection, unstable identification of subtle signs, high cost, large volume and energy consumption, etc.

[0005] (II) Technical solutions

[0006] The first aspect of the present disclosure provides an active interactive plant disease and pest inspection intelligent electronic eye system, which is installed on a mobile platform, the mobile platform comprising: a mobile platform chassis, a gimbal mast, a depth camera, an infrared camera, an active near-infrared lighting ring, a mechanical arm, a flexible actuator, a computing unit, a machine body, a communication and positioning unit, and an anti-glare shade; wherein the machine body is arranged on the mobile platform chassis, and the machine body is provided with a power supply; the mechanical arm and the computing unit are respectively arranged at both ends of the upper surface of the machine body, the first end of the mechanical arm is connected to the flexible actuator, and the second end of the mechanical arm close to the flexible actuator is provided with the gimbal mast; the communication and positioning unit is arranged on the computing unit; the anti-glare shade, the active near-infrared lighting ring, the depth camera and the infrared camera are all arranged on the gimbal mast.

[0007] According to an embodiment of the present disclosure, the gimbal mast comprises a camera fixing area, a flexible actuator fixing area, and a flexible actuator torque area; the camera fixing area is arranged at one end of the gimbal mast, the flexible actuator fixing area is arranged at the other end of the gimbal mast, and the flexible actuator torque area is arranged in the middle of the flexible actuator fixing area.

[0008] According to an embodiment of the present disclosure, the mobile platform chassis comprises a wheeled, tracked, footed or rail / slide platform, or a composite platform formed by combining any two or more of the above; the infrared camera comprises a near-infrared camera and a thermal infrared camera; the depth camera and the infrared camera are calibrated by internal parameters, external parameters and depth-infrared registration; the active near-infrared illumination ring and the anti-glare shade are used to improve the imaging quality in low-illumination and strong-reflective light scenes, so as to reduce the influence of ambient light and mirror highlights; the flexible actuator comprises a compliant joint and a flexible material finger tip, and combines force / torque sensing to realize force-position hybrid control to limit leaf surface contact force; the power supply arranged on the machine body comprises a power supply and a battery; the communication and positioning unit comprises a global satellite navigation system / real-time dynamic carrier phase difference technology module, an inertial measurement unit, and a fifth-generation mobile communication technology / special network module.

[0009] According to an embodiment of the present disclosure, further comprising: an upper computer and a cloud platform; the upper computer establishes a link with the computing unit, for real-time monitoring and parameter configuration, receiving event alarms and issuing instructions such as emergency stop and additional sampling; the cloud platform provides a pest knowledge base and version distribution service, a space-time report and asset / permission management function, supports cross-terminal data aggregation and federal / incremental learning mechanism; the computing unit carries an algorithm platform function, for completing depth-infrared calibration and registration, angle-distance-illumination geometric normalization, target detection / instance segmentation and insect tracking, three-dimensional reconstruction and visual field analysis, quality evaluation and closed-loop additional sampling decision, curved surface area and severity quantification, and evidence bag generation with space-time labels, and returns summaries / alarm to the upper computer and the cloud platform.

[0010] According to the embodiment of the present disclosure, the depth camera, infrared camera, active near-infrared illumination ring, and anti-glare lens hood are all mounted on the camera fixing area; the upper surface of the camera fixing area is provided with a hole array and positioning structure for mounting, including mounting holes, and the front edge of the camera fixing area is provided with an annular step / slot; the depth camera and the infrared camera are respectively positioned by screwing through the mounting holes; the active near-infrared illumination ring and the anti-glare lens hood are connected to the camera fixing area in a bayonet / screw thread manner and fixed to the annular step / slot; the mechanical arm is mounted to the flexible actuator fixing area in a detachable hinged manner, the flexible actuator torque area constitutes a series of elastic / compliant joints, including torque sensors and elastic elements for limiting and measuring leaf turning torque; the flexible actuator is fixed to one end of the mechanical arm and connected to the mechanical arm in a detachable hinged manner, and a controllable torque / compliant structure is formed at the flexible actuator torque area to achieve safe leaf turning and rebound; the cables required by the mechanical arm and the flexible actuator and the depth camera, infrared camera, active near-infrared illumination ring, and anti-glare lens hood are routed along the gimbal mast, and are transitioned through the hollow adapter at the flexible actuator torque area or the cable winding ring.

[0011] According to the embodiment of the present disclosure, the system installs a depth camera and an infrared camera on the camera fixing area of the gimbal mast, installs a flexible actuator through a mechanical arm at the flexible actuator fixing area, and realizes compliant force limiting through the flexible actuator torque area; after obtaining multi-modal images and depth along a preset path, the infrared image is subjected to angle-distance-illumination multi-dimensional geometric normalization and weather / clarity preprocessing based on the estimated leaf normal and camera-target distance; a multi-modal feature vector containing geometric features, thermal radiation features, texture features, and time sequence features is constructed, instance segmentation and lesion / worm detection are performed in the leaf / fruit area; when the visibility or confidence is lower than a first set threshold, a re-sampling pose is generated according to three-dimensional visual field analysis and uncertainty evaluation, the gimbal is fine-tuned or the mechanical arm is actuated to turn / turn the leaves to expose the back of the leaves and re-sample; short-time sequence data is registered and time-domain enhanced to improve the detectability of micro-motion targets; at least the lesion area ratio and the minimum distance to the main leaf vein are quantified on the reconstructed curved surface, and an evidence bag with spatiotemporal labels is generated for display and decision-making on the host computer / cloud platform.

[0012] According to an embodiment of the present disclosure, in order to solve the imaging defects caused by high-speed movement and occlusion of living insects, the system is also used for: event-driven acquisition: a high-frame-rate near-infrared channel is composed of the infrared camera and the active near-infrared illumination ring, real-time detection of motion saliency area, combined with instance segmentation and lesion quantification results, and according to the body segment proportion criterion output by the pre-trained skeleton network model, the evidence mode is triggered; Multi-view geometric aggregation: control the quick switching of small baseline view poses of the pan-tilt mast according to the field of view analysis result, and align multiple frames through depth guided registration; Physical consistent reconstruction: in the calculation unit, the geometric normalization processing and the time domain micro-motion amplification algorithm are fused, and the joint deblurring and super-resolution reconstruction are performed; Closed loop forensics: when the occlusion causes the clarity to be lower than the second set threshold, the mechanical arm and the flexible actuator are dispatched to flip and recapture, and the evidence package with three-dimensional scale and spatiotemporal label is output.

[0013] According to an embodiment of the present disclosure, the computing unit is used for three-dimensional reconstruction and pose planning, geometric normalization image processing, clear image reconstruction, instance segmentation, and pest lesion identification and quantification, and is used for edge computing and learning update; the computing unit performs angle-distance normalization processing on the infrared intensity based on the leaf normal obtained by depth estimation and the camera-target distance, and triggers active resampling when the visibility is lower than a first set threshold; the computing unit constructs a point cloud / grid and performs a viewable domain analysis, performs coverage measurement and blind area analysis, and generates a 360° pose planning path for main patrol and supplementary shooting; the geometric normalization includes normalization and highlight suppression processing on the infrared intensity according to the leaf normal, the camera-target distance and the active illumination intensity; depth registration, time domain filtering and micro-motion amplification are performed on short-time sequence images obtained after leaf plucking / turning operation, so as to enhance the detection sensitivity of insect micro-motion features; the instance segmentation outputs a single leaf / single fruit mask and its normal in a three-dimensional space, so as to obtain a curved surface area, a lesion area and a severity score; the system supports online incremental learning, which is used for updating the recognition model parameters according to a small amount of labeled samples at the edge, and the system can integrate a micro-quantity fixed-point spraying unit, which is used for implementing precise pesticide application or labeling on the positioned lesion area; an environment map is constructed based on depth data and coverage / blind area evaluation is performed, and a 360° main patrol and supplementary shooting path is generated; the leaf normal and the camera-target distance are estimated by depth estimation, and the infrared image is geometrically normalized; instance segmentation is performed in a three-dimensional space and the visibility is evaluated; when the leaf back visibility is lower than a first set threshold, the mechanical arm and the flexible actuator are controlled to pluck / turn the leaves and resample; pest lesion identification and quantification are performed on the normalized infrared / depth fusion features and are given geographical labeling; closed-loop supplementary shooting and optional fixed-point treatment are triggered according to the blind area statistical results; the insect motion saliency is detected through a high-frame-rate near-infrared channel, and when the confidence exceeds a third set threshold, an evidence collection mode is started; based on point cloud viewable domain analysis, a collection path of at least three small-baseline view poses is generated; multi-angle frames are projected to a unified coordinate system by using depth-infrared registration external parameters, and geometric normalization and highlight suppression are performed; high-definition images of insects are reconstructed through joint time domain deblurring and multi-frame super-resolution reconstruction technology; when the leaf edge shielding causes the clarity to be lower than a second set threshold, the flexible actuator is controlled to perform a leaf turning action and resample; the antenna and segment texture features of the insect body are enhanced based on micro-motion amplification; an evidence package containing three-dimensional scale estimation, spatiotemporal position label and hash signature is output; the model in the system takes multi-modal input images as the starting point, performs size normalization on the input images and records mapping parameters; according to the leaf normal estimated by the depth field and the camera-target distance and the illumination / exposure information, angle-distance-illumination geometric normalization is performed on the infrared channel to obtain an intensity image independent of the viewing angle; the normalized image is subjected to weather / visibility discrimination and restoration to obtain an image for subsequent analysis;The multi-channel edge and texture lightweight features are constructed and fused with temperature / reflective channels in a multi-branch encoder to generate a feature representation for detection / segmentation; a target detection / instance segmentation module based on the feature representation outputs instance regions and confidence, and calculates a quantitative index under three-dimensional / geometric constraints, while imaging quality and uncertainty are evaluated; when the results meet a first preset condition, the segmentation and quantitative results are output, and when the first preset condition is not met, instructions are fed back to a pose planning / active supplementary sampling module to trigger gimbal fine tuning or leaf re-sampling.

[0014] The second aspect of the present disclosure provides a method based on active interactive pest inspection intelligent electronic eye system, comprising: step 1. Device installation and initialization; Step 1.1 Mechanism level and installation: arranging the body on the mobile platform chassis, installing the mechanical arm and the computing unit at both ends of the body respectively, fixing the depth camera, infrared camera, active near-infrared illumination ring and anti-glare shade in the camera fixing area, detachably connecting the mechanical arm to the flexible actuator fixing area, installing the flexible actuator at the end of the mechanical arm, and forming an adjustable compliant and force limiting connection between the gimbal mast and the mechanical arm through the flexible actuator torque area; Step 1.2 Calibration and synchronization: completing the internal and external parameter calibration and re-projection of the depth camera and infrared camera in the computing unit, establishing a consistent coordinate system of point cloud and infrared intensity; synchronizing the depth camera, infrared camera, active near-infrared illumination ring, communication and positioning unit with time stamp, and recording the mapping parameters from pixels to physical size; Step 1.3 Stable orientation and illumination initialization: turning on the stable orientation control of the gimbal mast, decoupling the chassis pitch / roll disturbance; setting the active near-infrared illumination ring to the low contrast mode of oblique incidence, and using the anti-glare shade to suppress the highlight of fruit wax / dew, thereby providing high signal-to-noise ratio input for subsequent fruit surface texture and distance field acquisition; Step 2. Main patrol collection and quality control; generating a 360° main patrol pose set along the rows of fruit trees, and collecting depth D and infrared / near-infrared images I after the mobile platform chassis and the gimbal mast cooperate to the target pose; performing online quality control of definition, overexposure / underexposure, glare, raindrop shielding, etc., and re-collecting the image frames that do not meet the second preset condition in real time, thereby forming a multi-view pose image group and a pose trajectory of the fruit; Step 3. Geometric normalization, registration and surface reconstruction; based on the depth field, estimating the distance and normal of the fruit surface to the camera, performing angle-distance-illumination geometric normalization on the infrared intensity, and weakening the radiation distortion caused by the viewing angle / distance; performing pixel-level registration and point cloud-intensity fusion by using the calibration results of step 1.2; calling the instance segmentation backbone network, outputting the single fruit mask, and performing curvature adaptive surface reconstruction and normal consistency filtering under the constraint of the single fruit mask, and completing the parametric development of the fruit surface, thereby providing a scale-unified measurement domain for subsequent area and circumferential quantization; Step 4. Disease spot identification and three-dimensional quantization; refining the segmentation and artifact suppression of the disease spot candidates in the reconstructed surface / development domain, outputting the disease spot pixel-level mask and back-projecting it to the real surface, obtaining the real surface area and area ratio of the disease spot; combining the circumferential angle distribution and radiation difference, calculating the maximum feature size and circumferential unevenness index, and generating a disease spot quantization report for the fruit; meanwhile, retaining the pose, time stamp and calibration version matched with each quantization result for evidence closed loop and traceability; Step 5.Blind area identification, supplementary acquisition closed loop and result delivery; visible coverage volume is maintained in local grid / voxel map, visibility of target fruit surface is estimated according to field of view, occlusion and normal, at least the inside of fruit table / branch occluded area is marked as blind area and triggers secondary pose re-planning, redundant imaging around multi-baseline view is obtained; when network connectivity does not meet the third preset condition, interpretation and evidence package generation are preferentially completed at the edge, and original and derived elements are uploaded to the cloud platform in an intermittent synchronous manner, the upper computer displays the disease spot heat map and quantitative index of the current plant / current row in real time to guide flower and fruit protection, thinning or disease plant marking and disposal, realizing stable, comparable and traceable three-dimensional quantification of fruit disease spots in the scene of "weak network-strong occlusion-high reflectivity".

[0015] According to the embodiments of the present disclosure, the step 2 of 360° main patrol pose set specifically includes the following steps: firstly, self-positioning and mapping are completed through laser / vision odometer and global satellite navigation system fusion, time synchronization is performed on the depth camera, infrared camera and active near-infrared lighting ring, and internal and external hand-eye calibration parameters are loaded; a 360° main patrol pose set around the target plant is generated in the reachable space of the mechanical arm and the gimbal, and multi-modal data is sequentially collected, online quality control of clarity, overexposure and raindrop occlusion is performed, and image frames that do not meet the second preset condition are automatically re-acquired; the calculation unit estimates the leaf normal and camera-target distance according to the depth, performs geometric normalization and highlight suppression of the infrared image driven by angle-distance-illumination intensity, and performs non-uniformity correction / emissivity correction and haze / denoising clear image reconstruction; the local point cloud / grid is obtained by fusing multi-pose depth frames, the coverage and blind area distribution are obtained by carrying out visible domain analysis, and the detection and instance segmentation of leaves / fruits / stems and suspicious insect bodies, leaf front / back discrimination and visibility / uncertainty evaluation are completed under three-dimensional constraints; when the leaf back visibility is lower than the first set threshold or the identification uncertainty exceeds the tolerance, the system triggers closed loop supplementary acquisition: the gimbal fine tuning and the mechanical arm pose are solved based on the blind area distribution, the contact point and the contact direction are determined, and the virtual wall / safety boundary is set; the flexible actuator completes the leaf turning according to the approaching-pressing leaf-releasing three-stage trajectory under the cooperation of the support force PID channel and the torque PID channel, prevents damage in combination with the friction map / contact surface monitoring, and quickly re-samples the leaf back under local supplementary lighting; depth registration, time domain filtering and micro-motion amplification are performed on the supplementary acquisition short time sequence, insect detection and automatic tracking are realized in combination with the light flow / trajectory and other time sequence features, and the disease spot surface area, the minimum distance to the main leaf vein and the severity index are quantified on the reconstructed surface; the system generates an evidence package from the multi-modal image, the reconstructed grid, the trajectory ID, the pose and the timestamp, and writes it into the simultaneous localization and mapping / global satellite navigation system anchor point; when the coverage reaches the fourth set threshold, the inspection of the plant is ended, and when the fourth set threshold is not reached, the "main patrol-evaluation-supplementary acquisition" closed loop is formed by continuing the supplementary acquisition planning.

[0016] (Three) beneficial effects

[0017] 1. Proactive interaction and closed-loop coverage reduce missed detections and provide physical consistency quantification. 3D reconstruction and visual field analysis quantify coverage and blind spots, triggering the mechanism to perform leaf picking / turning for supplementary sampling, forming a "main inspection - assessment - supplementary sampling" closed loop; output indicators such as lesion surface area and minimum distance to the main leaf vein on the reconstructed surface, taking into account the observability and quantitative consistency of the underside of the leaf / occluded areas.

[0018] 2. Stable identification of subtle signs under complex lighting / viewing conditions. Through depth-infrared calibration and registration + geometric normalization + active illumination and sharp image reconstruction + short-term depth registration / micro-motion magnification, the effects of lighting changes, angles, and highlights / haze are significantly reduced, and early lesions, leaf back insect eggs, and small insects are stably displayed.

[0019] 3. Low cost, easy deployment, and sustainable operation and maintenance. Adopting depth + infrared technology instead of hyperspectral imaging significantly reduces cost, size, and power consumption, facilitating large-scale mobile inspections. It supports edge-based online incremental learning for rapid adaptation to new crops / seasons / scenarios and generates evidence packages with spatiotemporal anchors, facilitating verification and tracing of quarantine / intrusion monitoring.

[0020] It can achieve all-weather, low-cost, and practical automatic inspection and point-by-point investigation in environments such as greenhouses, orchards, and gardens, which is significantly better than existing solutions that rely solely on visible light or passive photography.

[0021] This disclosure addresses at least the following technical bottlenecks in pest and disease inspection in densely vegetated environments such as greenhouses and orchards:

[0022] 1. The distortion of infrared features caused by the dependence of viewpoint / distance and the uncertainty of illumination results in low detectability of weak lesions and small insects;

[0023] 2. Systemic missed detections due to canopy shading and the inability to see the underside of leaves;

[0024] 3. Passive single-scan technology lacks quantification of "coverage-blind spots" and a closed loop for re-shooting, and cannot guarantee full coverage at the plant / leaf level. Attached Figure Description

[0025] To gain a more complete understanding of this disclosure and its advantages, reference will now be made to the following description taken in conjunction with the accompanying drawings, wherein:

[0026] Figure 1 This schematic diagram illustrates the overall structural block diagram of an active interactive intelligent electronic eye system for pest and disease inspection provided in an embodiment of the present disclosure;

[0027] Figure 2 This illustration schematically shows a structural diagram of the main components of an active interactive intelligent electronic eye system for pest and disease inspection provided in an embodiment of this disclosure;

[0028] Figure 3A disease and pest inspection flowchart provided by the embodiments of the present disclosure is schematically shown.

[0029] Figure 4 A depth-infrared co-field-of-view image provided by the embodiments of the present disclosure is schematically shown.

[0030] Figure 5 A 360° coverage-blind area closed-loop scanning and pose planning flowchart provided by the embodiments of the present disclosure is schematically shown.

[0031] Figure 6 A gimbal mast structure schematic diagram provided by the embodiments of the present disclosure is schematically shown.

[0032] Figure 7 An active leaf turning end execution schematic diagram provided by the embodiments of the present disclosure is schematically shown.

[0033] Figure 8 A fruit / leaf / pest and disease image processing schematic diagram using instance segmentation and lesion quantification result visualization algorithm provided by the embodiments of the present disclosure is schematically shown.

[0034] Figure 9 A skeleton network model diagram of instance segmentation and lesion quantification result provided by the embodiments of the present disclosure is schematically shown.

[0035] Figure 10 An insect automatic tracking and closed-loop resampling flowchart provided by the embodiments of the present disclosure is schematically shown.

[0036] Figure 11 A structure schematic diagram of a depth camera, an infrared camera, an active near-infrared illumination ring and an anti-glare shade of a camera fixing area provided by the embodiments of the present disclosure is schematically shown.

[0037] Legend of reference signs:

[0038] 1 - mobile platform chassis; 2 - gimbal mast; 3 - depth camera; 4 - infrared camera; 5 - active near-infrared illumination ring; 6 - mechanical arm; 7 - flexible actuator; 8 - computing unit; 9 - body; 10 - communication and positioning unit; 11 - anti-glare shade; 210 - camera fixing area; 220 - flexible actuator fixing area; 221 - flexible actuator torque area. DETAILED DESCRIPTION

[0039] Embodiments of the present disclosure will be described below with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. However, it will be apparent to those skilled in the art that the embodiments can be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques have been omitted to avoid unnecessarily obscuring the concepts of the present disclosure.

[0040] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. As used herein, the term "includes" and tautological expressions thereof, such as "including," "includes," "include," "contains," "containing," "contain" and other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0041] All terms used herein, including technical and scientific terms, have the meanings commonly understood by one of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having meanings consistent with the context of the specification, and should not be interpreted in an idealized or overly formal manner.

[0042] It has been found through research that there is an urgent need for a low-cost, mobile system that can stably identify subtle signs under complex lighting and shading conditions, has the ability of active interaction (such as leaf pulling / turning), and can be applied to multiple scenarios such as agricultural production, plant quarantine, plant protection, and invasive pest monitoring.

[0043] As shown in Figure 1 , Figure 2 and Figure 11 , the present disclosure provides an active interactive disease and pest inspection intelligent electronic eye system, which can be installed on a mobile platform, the mobile platform comprising a mobile platform chassis 1, a gimbal mast 2, a depth camera 3, an infrared camera 4, an active near-infrared lighting ring 5, a mechanical arm 6, a flexible actuator 7, a computing unit 8, a body 9, a communication and positioning unit 10, and an anti-glare shade 11.

[0044] The body 9 is arranged on the mobile platform chassis 1, and a power supply is arranged on the body 9. The mechanical arm 6 and the computing unit 8 are respectively arranged at the two ends of the upper part of the body 9. The communication and positioning unit 10 is arranged on the computing unit 8. The flexible actuator 7 is connected to the first end of the mechanical arm 6. The gimbal mast 2 is arranged on the mechanical arm 6 at the second end opposite to the first end and close to the flexible actuator 7. The anti-glare shade 11, the active near-infrared lighting ring 5, the depth camera 3, and the infrared camera 4 are all arranged on the gimbal mast 2.

[0045] The mobile platform chassis 1 comprises a wheeled, tracked, footed, or rail / slide platform, or a composite platform in the form of any two or more of the above.

[0046] The infrared camera 4 includes a near-infrared camera and a thermal infrared camera.

[0047] The depth camera 3 and the infrared camera 4 are calibrated by intrinsic calibration, extrinsic calibration and depth-infrared registration.

[0048] The active near-infrared illumination ring 5 and the anti-glare hood 11 are used to improve the imaging quality in low-illumination and strong-reflective scenes, so as to reduce the influence of ambient light and mirror highlights.

[0049] The flexible actuator 7 includes compliant joints and flexible material finger ends, and combines force / torque sensing to realize force-position hybrid control to limit leaf surface contact force.

[0050] The body 9 includes power supply and battery supply.

[0051] The communication and positioning unit 10 includes a global navigation satellite system (GNSS) / real-time kinematic carrier phase difference technology module (RTK), an inertial measurement unit (IMU) and a fifth generation mobile communication technology (5G) / dedicated network module.

[0052] As shown in Figure 6 The gimbal mast 2 includes a camera fixing area 210, a flexible actuator fixing area 220 and a flexible actuator torque area 221.

[0053] The camera fixing area 210 is arranged at one end of the gimbal mast 2, the flexible actuator fixing area 220 is arranged at the other end of the gimbal mast 2, and the flexible actuator torque area 221 is arranged in the middle of the flexible actuator fixing area 220.

[0054] The depth camera 3, the infrared camera 4, the active near-infrared illumination ring 5 and the anti-glare hood 11 are all mounted on the camera fixing area 210. The upper surface of the camera fixing area 210 is provided with a hole array and a positioning structure for mounting, including mounting holes; the front edge is provided with an annular step / slot. The depth camera 3 and the infrared camera 4 are respectively positioned by screwing through the mounting holes; the active near-infrared illumination ring 5 and the anti-glare hood 11 are connected with the camera fixing area 210 in a bayonet / screw thread manner, and are fixed in the annular step / slot.

[0055] The mechanical arm 6 is detachably hinged to the flexible actuator fixing area 220, and the flexible actuator torque area 221 constitutes a series elastic / compliant joint (including torque sensing and elastic elements) for limiting and measuring leaf turning torque. The flexible actuator 7 is fixed at one end of the mechanical arm 6, and is detachably hinged to the mechanical arm 6, and forms a controllable torque / compliant structure at the flexible actuator torque area 221 to realize safe leaf turning and rebound.

[0056] The cables required for the actuator mechanism mechanical arm 6 and the flexible actuator 7 and the camera mechanism depth camera 3, infrared camera 4, active near-infrared illumination ring 5, and anti-glare shade 11 are routed along the gimbal mast 2, and are transitioned through the hollow adapter at the flexible actuator torque zone 221 or cable winding ring to ensure joint movement and cable safety.

[0057] The gimbal mast 2 is used for decoupling the mobile platform attitude and the flexible actuator 7 attitude; the gimbal mast 2 isolates and compensates for changes in the attitude of the mobile platform body, so that the spatial attitude of the onboard sensors of the depth camera 3, infrared camera 4, and flexible actuator 7 mounted thereon is independently controlled by the gimbal. Even if the mobile platform produces pitch, roll, or vibration during travel, the sensor / optical axis remains pointing in the preset direction or changes as instructed, thereby ensuring stable imaging and safe leaf flipping contact.

[0058] The computing unit 8 is used for three-dimensional reconstruction and pose planning, geometric normalized image processing, clear image reconstruction, instance segmentation, and pest / lesion identification and quantification, and is used for edge computing and learning updates.

[0059] The computing unit 8 performs angle-distance normalization processing on the infrared intensity based on the leaf normal obtained by depth estimation and the camera-target distance, to improve the identification robustness, and triggers active re-sampling when the visibility is insufficient, i.e., below a first set threshold.

[0060] The computing unit 8 constructs a point cloud / grid and performs a viewable domain analysis, performs coverage measurement and blind area analysis, and generates a 360° pose planning path for main patrol and re-shooting.

[0061] The geometric normalization includes normalization and highlight suppression processing of the infrared intensity according to the leaf normal, camera-target distance, and active illumination intensity.

[0062] Depth registration, time domain filtering, and micro-motion amplification are performed on the short-time sequence images obtained after the leaf flipping / folding operation, to enhance the detection sensitivity of insect micro-motion features.

[0063] The instance segmentation outputs a single leaf / single fruit mask and normal in three-dimensional space to obtain the curved surface area, lesion area, and severity score.

[0064] The system supports online incremental learning, which is used to update the identification model parameters at the edge end according to a small amount of labeled samples. The system can integrate a micro-amount of fixed-point spraying unit, which is used to implement precise pesticide application or labeling on the positioned lesion area.

[0065] Construct environment map and coverage / blind area evaluation based on depth data to generate 360° main patrol and supplementary shooting path; use depth estimation to normalize geometry and distance to infrared image; perform instance segmentation in three-dimensional space and evaluate visibility; control robotic arm 6 and flexible actuator 7 to flip leaves when leaf back visibility is below threshold; identify and quantify insect pests / lesions on normalized infrared / depth fusion features and assign geographic labels; trigger closed-loop supplementary shooting and optional point treatment according to blind area statistics.

[0066] Detect insect motion saliency through high-frame-rate near-infrared channel, and start evidence collection mode when confidence exceeds threshold (third set threshold); based on point cloud viewable area analysis, generate at least three small baseline view pose collection paths; use depth-infrared registration external parameters to project multi-angle frames into a unified coordinate system, and perform geometric normalization and highlight suppression; reconstruct high-definition insect images through joint time-domain deblurring and multi-frame super-resolution reconstruction technology; when leaf edge occlusion causes clarity to be below second set threshold, control flexible actuator 7 to perform leaf flipping action and re-sampling.

[0067] Based on micro-motion amplification to enhance insect antennae and segment texture features; output evidence package containing three-dimensional scale estimation, spatiotemporal position label and hash signature.

[0068] As shown in Figure 1 An active interactive pest and disease patrol intelligent electronic eye system includes a host computer and a cloud platform, the host computer and the computing unit 8 establish a link for real-time monitoring and parameter configuration, receiving event alarm and issuing supplementary shooting / emergency stop instructions; the cloud platform provides pest and disease knowledge base and version distribution, cross-terminal data aggregation and federal / incremental learning, spatiotemporal report and asset / permission management. The computing unit 8 carries the algorithm platform function, completes depth-infrared calibration and registration, angle-distance-illumination geometric normalization, target detection / instance segmentation and insect tracking, three-dimensional reconstruction and viewable area analysis, quality evaluation and closed-loop supplementary shooting decision, curved surface area and severity quantification and evidence package generation with spatiotemporal label, and returns summary / alarm to the host computer and the cloud platform.

[0069] As shown in Figure 3 and Figure 6As shown, the system installs a depth camera 3 and an infrared camera 4 in the camera fixing area 210 of the gimbal mast 2, installs a flexible end 7 in the flexible actuator fixing area 220 through a mechanical arm 6, and realizes compliant force limiting through the torque area 221; after obtaining multi-modal images and depth along the preset path, the infrared image is angle-distance-illumination geometry normalized based on the depth estimated leaf normal and camera-target distance, and necessary weather / clarity preprocessing is completed; a multi-modal feature vector containing geometric features, thermal radiation features, texture features and time sequence features is constructed, instance segmentation and lesion / worm detection are performed in the leaf / fruit area; when the visibility or confidence is insufficient, i.e. lower than the first set threshold, the three-dimensional visual field analysis and uncertainty evaluation are generated to generate a supplementary sampling pose, drive the gimbal fine adjustment or the robot-arm (mechanical arm) to execute the leaf stirring / leaf turning to expose the back of the leaf and resample; the short time sequence data is registered and time domain enhanced to improve the detectability of micro-motion targets; finally, the lesion area ratio, the minimum distance to the main leaf vein and other quantities are quantified on the reconstructed curved surface, and an evidence bag with spatiotemporal labels is generated for display and decision-making on the host computer / cloud (i.e. cloud platform).

[0070] As shown in Figure 4 Depth-infrared registration, the present disclosure uses a common field of view internal calibration board or spontaneous texture / structure light features to obtain the extrinsic parameter set and the intrinsic parameter set between the depth camera 3 and the infrared camera 4.

[0071] Distortion correction and re-projection are completed in the computing unit 8, realizing pixel-level registration and point cloud-infrared intensity fusion.

[0072] After depth-infrared registration, three-dimensional positioning of lesions, real scale estimation and leaf / fruit surface unfolding measurement can be realized in the same coordinate system.

[0073] As shown in Figure 5As shown, the active interactive pest patrol intelligent electronic eye system of the present disclosure first completes self-localization and mapping through laser / visual odometry and GNSS fusion, time-synchronizes the depth camera 3, infrared camera 4, and active near-infrared illumination ring 5, and loads internal and external hand-eye calibration parameters; then generates a 360° main patrol pose set around the target plant within the reachable space of the machine / arm / gimbal, and sequentially collects multi-modal data, performs online quality control of sharpness, overexposure, and raindrop shielding, and automatically re-samples the image frames that do not meet the second preset condition. The computing unit 8 estimates the leaf normal and camera-target distance according to the depth, performs geometric normalization and highlight suppression driven by angle-distance-illumination intensity on the infrared image, and performs non-uniformity correction (NUC) / emissivity correction and dehazing / denoising clear image reconstruction; fuses multi-pose depth frames to obtain local point clouds / grids, conducts a viewable area analysis to obtain coverage and blind area distribution, and completes detection and instance segmentation of leaves / fruits / stems and suspicious insect bodies, leaf front / back discrimination, and visibility / uncertainty evaluation under three-dimensional constraints. When the leaf back visibility is lower than the first set threshold or the recognition uncertainty rises beyond the tolerance, the system triggers closed-loop re-sampling: based on the blind area distribution, the gimbal fine-tuning and machine arm pose are solved, the contact point and contact direction are determined, and a virtual wall / safety boundary is set; the flexible actuator 7 completes leaf turning according to the "approach-press leaf-release" three-stage trajectory under the cooperation of the support force PID channel (pressure sensor-pressure drive closed loop) and the torque PID channel (torque sensor-motor drive closed loop), and prevents damage in combination with the friction map / contact surface monitoring, and completes rapid re-sampling of the leaf back under local supplementary lighting. Depth registration, time domain filtering, and micro-motion amplification are performed on the re-sampling short time sequence, insect detection and automatic tracking are realized in combination with light flow / trajectory and other time sequence features, and the disease spot surface area, minimum distance to the main leaf vein, and severity index are quantified on the reconstructed surface; the system generates an evidence package of multi-modal images, reconstructed grids, trajectory ID, pose, and timestamp, and writes it into the simultaneous localization and mapping (SLAM) / GNSS anchor point; when the coverage reaches the fourth set threshold, the patrol of the plant is ended, and if the threshold is not reached, the re-sampling planning forms a "main patrol-evaluation-re-sampling" closed loop, and the filtering and environmental compensation, as well as the force / torque / speed limiting and emergency stop protection mechanisms are enabled throughout the process.

[0074] As shown in Figure 6 and Figure 9 The present disclosure maintains a visible coverage model in the local map, estimates the visibility of the target area according to the field of view parameters, occlusion relationship, and surface normal, marks low-visibility areas such as "leaf back" and "fruit table inner side" as blind areas, and triggers pose and path re-planning; the re-planning uses a heuristic multi-objective path planning and candidate view pose sampling method combined with local optimization, and is linked with the stabilization control mechanism of the gimbal mast 2, to realize stable view axis in motion and clear imaging in stability during dynamic acquisition.

[0075] As shown inFigure 7 As shown, the flexible actuator 7 is installed at the end of the mechanical arm 6 to implement the flicking / turning of the leaf edge through a high-friction contact interface to expose the back of the leaf, and the driving side cooperates with the force monitoring to realize overload protection; in cooperation with the depth camera 3 and the infrared camera 4, a visual-touch closed-loop criterion is established, which is based on target stability, turning angle estimation and shading / high light suppression degree to constrain the action, thereby improving the effective observation of the low-visibility area without damaging the leaf.

[0076] As shown, Figure 9 The model starts with a multi-modal input image, normalizes the size of the input image and records the mapping parameters; according to the leaf normal estimated by the depth field, the camera-target distance and the lighting / exposure information, the angle-distance-illumination geometric normalization is performed on the infrared channel to obtain an intensity map independent of the viewing angle; on this basis, weather / visibility discrimination and restoration are performed to obtain an image for subsequent analysis; then multi-channel edge and texture features are constructed and fused with temperature / reflection channels in a multi-branch encoder to generate a feature representation for detection / segmentation; the target detection / instance segmentation module based on the representation outputs instance regions such as lesions / insects / leaves and confidence, and calculates quantitative indicators such as area ratio and thermal contrast under three-dimensional / geometric constraints, while imaging quality and uncertainty are evaluated; when the results meet the preset criterion (i.e. the first preset condition), the segmentation and quantitative results are output, and when they do not meet the preset criterion, instructions are fed back to the pose planning / active resampling module to trigger gimbal fine-tuning or leaf flicking resampling.

[0077] As shown, Figure 10 To address the imaging defects caused by high-speed motion and occlusion of live insects, the system has the following capabilities:

[0078] 1. Event-driven acquisition: The high-frame-rate near-infrared channel composed of the infrared camera 4 and the active near-infrared lighting ring 5 detects motion saliency areas in real time, and combines instance segmentation and lesion quantification results to trigger evidence patterns according to the body segment proportion criterion of the pre-trained skeleton network model.

[0079] 2. Multi-view geometric aggregation: The gimbal mast 2 quickly switches to a small-baseline view pose according to the visual domain analysis results, and aligns multiple frames through depth-guided registration.

[0080] 3. Physical consistent reconstruction: In the computing unit 8, geometric normalization processing (suppressing distance / angle distortion) and time-domain micro-motion amplification algorithm are fused to perform joint deblurring and super-resolution reconstruction.

[0081] 4. Closed-loop forensics: When occlusion causes insufficient clarity, i.e. below the second set threshold, the mechanical arm 6 and the flexible actuator 7 are dispatched to flip the leaf for resampling, and finally an evidence package with three-dimensional scale and spatio-temporal label is output.

[0082] Example 1: Fruit inspection and lesion quantification.

[0083] In view of the objective ecological constraints of the fruit tree crown extending to the ground and the adjacent plant crowns intertwining with each other and accompanying weeds, the traditional walking or small wheeled platform is limited in passing between rows, the line of sight height and the field of view angle are strongly disturbed by the branches and leaves, it is difficult to form continuous imaging and repeatable quantification of fruit targets. The mobile recognition monitoring platform with the mobile platform chassis 1 as the main body, the computing unit 8 and the communication and positioning unit 10 integrated on the fuselage 9, the front end is provided with a gimbal mast 2: the depth camera 3, the infrared camera 4, the active near-infrared lighting ring 5 and the anti-glare shade 11 are installed in the camera fixing area 210, the flexible actuator 7 is fixed by the mechanical arm 6 in the flexible actuator fixing area 220, and the controllable compliance is formed in the torque area 221; thereby forming a multi-modal imaging and active leaf picking-up link to realize three-dimensional quantification of fruit lesions.

[0084] Step 1. Device installation and initialization, as shown in Figure 1 .

[0085] In step 1.1, the mechanism level and installation are carried out: the fuselage 9 is arranged on the mobile platform chassis 1, the depth camera 3, the infrared camera 4, the active near-infrared lighting ring 5 and the anti-glare shade 11 are fixed in the camera fixing area 210, the mechanical arm 6 is fixed in the flexible actuator fixing area 220 in a detachable hinged manner, the flexible actuator 7 is installed at the end of the mechanical arm 6, and the gimbal mast 2 and the mechanical arm 6 are connected through the flexible actuator torque area 221 to form an adjustable compliance and force limiting connection.

[0086] In step 1.2, calibration and synchronization are carried out: the internal and external parameter calibration and re-projection of the depth camera 3 and the infrared camera 4 are completed in the computing unit 8, as shown in Figure 4 , the point cloud coordinate system and the consistent coordinate system of the infrared intensity are established; the depth camera 3, the infrared camera 4, the active near-infrared lighting ring 5 and the communication and positioning unit 10 are time-stamped synchronized, and the mapping parameters from pixels to physical dimensions are recorded.

[0087] In step 1.3, the heading is stabilized and the illumination is initialized: the gimbal mast 2 is turned on to stabilize the heading and decouple the chassis pitch / roll disturbance; the active near-infrared lighting ring 5 is set to a low-contrast mode with oblique incidence, and the anti-glare shade 11 is used to suppress fruit wax / dew highlights, providing high signal-to-noise ratio input for subsequent fruit surface texture and distance field acquisition.

[0088] Step 2. Main patrol collection and quality control, as shown in Figure 5 .

[0089] Generate a 360° main patrol pose set along the fruit tree row, and the mobile platform chassis 1 and gimbal mast 2 cooperate to reach the target pose, that is, collect depth D and infrared / near-infrared images I after reaching the target pose; online perform quality control such as sharpness, overexposure / underexposure, reflection, raindrop shielding, etc., and if the image frame does not meet the second preset condition, it is immediately re-sampled to form a multi-view pose image group and a pose trajectory of the fruit.

[0090] Step 3. Geometric normalization, registration, and surface reconstruction, as shown in Figure 3 、 Figure 4 and Figure 8 .

[0091] Based on the depth field, estimate the distance and normal of the fruit surface to the camera, perform angle-distance-illumination geometric normalization on the infrared intensity, and weaken the radiation distortion caused by the viewing angle / distance; use the calibration results of step 1.2 for pixel-level registration and point cloud-intensity fusion; call an instance segmentation backbone network, as shown in Figure 8 , output a single fruit mask, and perform curvature adaptive surface reconstruction and normal consistency filtering under its constraint, and complete the parametric unfolding of the fruit surface, providing a scale-unified measurement domain for subsequent area and circumferential quantization.

[0092] Step 4. Lesion identification and three-dimensional quantization, as shown in Figure 8 .

[0093] Refine the segmentation and artifact suppression of lesion candidates in the reconstructed surface / unfolded domain, output the lesion pixel-level mask and back-project it to the real surface to obtain the real surface area and area ratio of the lesion; combine the circumferential angle distribution and radiation difference to calculate indicators such as maximum feature size and circumferential unevenness, and generate a fruit-oriented lesion quantization report. At the same time, the pose, timestamp, and calibrated version are retained with each quantization result for evidence closed loop and traceability.

[0094] Step 5. Blind area identification, supplementary sampling closed loop, and result delivery, as shown in Figure 1 and Figure 5 .

[0095] Maintain a visible coverage volume in the local grid / voxel map, estimate the visibility of the target fruit surface according to the field of view, occlusion, and normal, mark "inside the fruit table / branch occlusion area" as a blind area and trigger secondary pose re-planning (gimbal fine-tuning + chassis turning), and obtain redundant imaging from multiple baseline viewing angles; when network connectivity is insufficient, that is, does not meet the third preset condition, preferentially complete interpretation and evidence package generation at the edge, and upload the original and derived elements to the cloud platform in an intermittent synchronous manner, and the upper computer displays the lesion heat map and quantization indicators of the current plant / row in real time to guide flower and fruit protection, flower and fruit thinning, or disease plant marking and disposal, achieving stable, comparable, and traceable three-dimensional quantization of fruit lesions in "weak network-strong occlusion-high reflection" scenarios.

[0096] Embodiment 2: Leaf inspection and active leaf turning to expose leaf back.

[0097] For the scenario of leaf curtain layering, disease signs and insect eggs are mostly on the leaf back and are in the geometric shadow area for a long time, this embodiment configures a flexible actuator 7 at the end of the mechanical arm 6, and forms a limited torque and controllable flexibility contact-driven mechanism through the flexible actuator fixing area 220 and the flexible actuator torque area 221 of the gimbal mast 2. The flexible actuator torque area 221 provides compliance and force limiting when contacting the leaf margin, so that the flexible actuator 7 can complete the poking and achieve stable turning of not less than the preset angle under low contact stress; the flexible actuator 7 contact interface adopts a high-friction flexible surface to improve the adhesion and control ability of the wet and slippery leaf margin, and at the same time, through the built-in stress / current observation, overload discrimination and soft stop are realized to avoid damage to fragile tissues.

[0098] To ensure the cooperation of imaging and contact during leaf turning, the computing unit 8 performs robust fitting on the leaf margin curve and the main leaf vein based on the multi-modal instance segmentation result, and online solves the approaching posture and path consistent with the normal direction of the leaf back; the gimbal mast 2 maintains the included angle between the visual axis and the normal direction of the leaf back within a controllable interval, and cooperates with the oblique incidence and polarization anti-glare strategy of the active near-infrared illumination ring 5, significantly reduces the texture loss caused by corner gap reflection, and improves the effective contrast and detail fidelity within the window when the leaf back is exposed.

[0099] Within the time window of temporary exposure of the leaf back, the normal direction and curvature constraints provided by the depth camera 3 and the low-illumination texture captured by the infrared camera 4 are synchronously sent into the multi-branch head of the skeleton network to respectively generate leaf instance masks, lesion pixel-level masks, and geometric regression quantities such as thickness and curl; the system suppresses the inter-frame jitter caused by micro-vibration with time domain consistency regularization, and outputs quantified indicators facing the leaf in the leaf back expansion coordinate system, which at least include lesion area fraction, lesion density, and tunnel length and chlorophyll proxy index based on trajectory and texture, for subsequent disease course evaluation and grading.

[0100] When the first leaf turning does not meet the preset criteria of obstruction elimination or image clarity, the system quickly fine-tunes the gimbal angle and end contact pose according to the composite cost of coverage, energy consumption and contact risk, and preferentially selects the micro-baseline view angle with the highest gain to realize secondary re-sampling; the force / current observer at the end triggers soft stop and retreat when the abnormality rises, and automatically converges to a safe posture to wait for the next approach, ensuring continuous improvement of leaf back visibility and imaging quality under soft contact constraints.

[0101] In the park with limited network conditions, the edge end bundles the leaf back quantitative results with geographical / plant site labels to generate a leaf back disease sign heat map and visualize it in real time on the upper computer; when the cloud platform is connected, the system updates the domain adaptively by fusing multiple batches of multi-view data, continuously improving the adaptability of the leaf back recognition model to different crops and seasons.

[0102] Embodiment 3: Live insect identification, dynamic multi-angle imaging and high-definition reconstruction forensics.

[0103] As shown in Figure 10 , this embodiment is aimed at live insects with high-speed motion, short stay or occlusion in greenhouse, orchard and other scenarios. Based on the synchronization of gimbal, camera and calibration in Embodiment 1 and Embodiment 2, a special closed-loop forensics process is introduced, including event-driven acquisition, multi-angle micro-baseline imaging, physically consistent reconstruction and quality criterion. During the inspection process, the high-frame-rate near-infrared channel composed of infrared camera 4 and active near-infrared lighting ring 5 performs real-time motion saliency detection. Combined with the distance field provided by depth camera 3 and the pose constraints of communication and positioning unit 10, once the events such as antennae swing or body segment vibration are triggered, the system will quickly generate a number of small-baseline visual poses based on the visual field analysis results and drive the gimbal mast 2 to switch at high speed, forming a spatio-temporal microarray image group. When the insect body is occluded by the leaf edge, the mechanical arm 6 cooperates with the flexible actuator 7 to execute leaf pushing / turning to expand the visual field, while ensuring contact safety and imaging stability under the compliant force limiting protection of the flexible actuator torque zone 221. Figure 5

[0104] The collected multi-angle sequences complete multi-frame alignment guided by depth and angle-distance-illumination geometry normalization in the computing unit 8, and then carry out deblurring and enhanced reconstruction compatible with temporal consistency, improving the recognition of small body segments and textures. The system sets quality criteria such as sharpness and occlusion degree to screen the reconstruction results: if not up to standard, it will trigger closed-loop resampling through gimbal fine-tuning or leaf turning again until the preset threshold is met or the upper limit is reached; if it fails continuously, it will output failure logs and retain intermediate evidence to ensure the integrity of the forensics chain. After meeting the criteria, the reconstructed sequence is input into the model, first normalized in size and necessary visibility recovery, then output by the multi-branch lightweight feature fusion and target detection / instance segmentation module, including insect and related vegetation instances and their confidence, and the spatial scale of insect and lesion, insect residence time, instantaneous speed, trajectory curvature and micro-vibration characteristics under three-dimensional / geometric constraints; the possible semantic events such as feeding, egg laying and mating are classified, and the evidence package with three-dimensional scale estimation and spatio-temporal label (including high-definition still frames, short video clips, poses and timestamps) is generated, and the hash signature is used to ensure traceability. The upper computer draws insect density heat map and event distribution based on the evidence package in real time and correlates with disease spots and feeding traces, and the cloud platform conducts joint modeling on a cross-regional and multi-temporal basis, and outputs operation recommendations for prevention and control. Figure 9

[0105] ​​Those skilled in the art will appreciate that features recited in the various embodiments of the present disclosure can be combined and / or integrated in various combinations, even if such combinations have not been recited in the present disclosure. In particular, the features recited in the various embodiments of the present disclosure can be combined and / or integrated in various combinations without departing from the spirit and teachings of the present disclosure. All such combinations are within the scope of the present disclosure.

[0106] While the present disclosure has been illustrated and described with reference to certain exemplary embodiments thereof, it will be understood by those skilled in the art that various changes in form and details can be made therein without departing from the spirit and scope of the disclosure as defined by the appended claims and their equivalents. Therefore, the scope of the present disclosure should not be limited to the embodiments described herein but should be defined by the appended claims and their equivalents in connection with the full scope of equivalents thereof.

Claims

1. An active interactive pest inspection intelligent electronic eye system, characterized in that, The system is installed on a mobile platform, and the mobile platform comprises: A mobile platform chassis (1), a gimbal mast (2), a depth camera (3), an infrared camera (4), an active near-infrared lighting ring (5), a mechanical arm (6), a flexible actuator (7), a computing unit (8), a fuselage (9), a communication and positioning unit (10), and an anti-glare lens hood (11); wherein The fuselage (9) is arranged on the mobile platform chassis (1), and the fuselage (9) is provided with a power supply; The mechanical arm (6) and the computing unit (8) are respectively arranged at both ends of the upper surface of the fuselage (9), the first end of the mechanical arm (6) is connected to the flexible actuator (7), and the second end of the mechanical arm (6) close to the flexible actuator (7) is provided with the gimbal mast (2); The communication and positioning unit (10) is arranged on the computing unit (8); The anti-glare lens hood (11), the active near-infrared lighting ring (5), the depth camera (3), and the infrared camera (4) are all arranged on the gimbal mast (2); The gimbal mast (2) comprises a camera fixing area (210), a flexible actuator fixing area (220), and a flexible actuator torque area (221); The camera fixing area (210) is arranged at one end of the gimbal mast (2), the flexible actuator fixing area (220) is arranged at the other end of the gimbal mast (2), and the flexible actuator torque area (221) is arranged in the middle of the flexible actuator fixing area (220); A host computer and a cloud platform; The host computer and the computing unit (8) establish a link for real-time monitoring and parameter configuration, receiving event alarms, and issuing supplementary sampling / urgent stop instructions; The cloud platform provides a pest knowledge base and version distribution service, a space-time report and asset / permission management function, and supports cross-terminal data aggregation and federal / incremental learning mechanisms; The computing unit (8) carries an algorithm platform function, which is used to complete depth-infrared calibration and registration, angle-distance-illumination geometric normalization, target detection / instance segmentation and insect tracking, three-dimensional reconstruction and visual field analysis, quality evaluation and closed-loop supplementary sampling decision, curved surface area and severity quantification, and evidence bag generation with space-time labels, and returns summaries / alerts to the host computer and the cloud platform; The system installs a depth camera (3) and an infrared camera (4) in the camera fixing area (210) of the gimbal mast (2), installs a flexible actuator (7) in the flexible actuator fixing area (220) through a mechanical arm (6), and realizes compliance force limiting through the flexible actuator torque area (221); After obtaining multi-modal images and depth along a preset path, the leaf normal and camera-target distance estimated based on depth are used to perform angle-distance-illumination multi-dimensional geometric normalization on infrared images and complete weather / clarity preprocessing; a multi-modal feature vector containing geometric features, thermal radiation features, texture features, and time sequence features is constructed, and instance segmentation and disease spot / insect body detection are performed in leaf / fruit regions; When the visibility or confidence is lower than the first set threshold, the supplementary pose is generated according to the three-dimensional visual field analysis and uncertainty evaluation, and the gimbal fine adjustment or the mechanical arm (6) is driven to perform the leaf stirring / turning to expose the leaf back and re-sampling; The short-time sequence data is registered and time-domain enhanced to improve the detectability of micro-motion targets; at least the lesion area ratio and the minimum distance to the main leaf vein are quantified on the reconstructed surface, and an evidence bag with space-time labels is generated for display and decision-making on the host computer / cloud platform.

2. The active interactive intelligent electronic eye system for disease and pest inspection according to claim 1, characterized in that, The mobile platform chassis (1) includes a wheeled, tracked, footed, or rail / slide platform, or a composite platform formed by combining any two or more of the above; The infrared camera (4) includes a near-infrared camera and a thermal infrared camera; The depth camera (3) and the infrared camera (4) are calibrated by internal parameters, external parameters, and depth-infrared registration; The active near-infrared illumination ring (5) and the anti-glare light shield (11) are used to improve the imaging quality in low-light and strong-light scenes, to reduce the influence of environmental light and mirror highlights; The flexible actuator (7) includes a compliant joint and a flexible material finger tip, and combines force / torque sensing to realize force-position hybrid control to limit leaf surface contact force; The power supply provided on the body (9) includes a power supply and a battery; The communication and positioning unit (10) includes a global satellite navigation system / real-time dynamic carrier phase difference technology module, an inertial measurement unit, and a fifth-generation mobile communication technology / special network module. 3.The active interactive intelligent electronic eye system for disease and pest inspection according to claim 1, wherein, The depth camera (3), the infrared camera (4), the active near-infrared illumination ring (5), and the anti-glare light shield (11) are installed on the camera fixing area (210); The upper surface of the camera fixing area (210) is provided with a hole array and a positioning structure for installation, including mounting holes, and the front edge of the camera fixing area (210) is provided with an annular step / slot; The depth camera (3) and the infrared camera (4) are respectively positioned by the mounting holes; The active near-infrared illumination ring (5) and the anti-glare light shield (11) are connected to the camera fixing area (210) in a bayonet / screw thread manner and are fixed to the annular step / slot; The mechanical arm (6) is detachably connected to the flexible actuator fixing area (220), and the flexible actuator torque area (221) constitutes a series of elastic / compliant joints, including torque sensors and elastic elements, for limiting and measuring the turning torque; The flexible actuator (7) is fixed to one end of the mechanical arm (6) and is detachably connected to the mechanical arm (6), and a controllable torque / compliant structure is formed at the flexible actuator torque area (221) to realize safe turning and rebound; The cables required by the mechanical arm (6) and the flexible actuator (7) and the depth camera (3), the infrared camera (4), the active near-infrared illumination ring (5), and the anti-glare light shield (11) are routed along the gimbal mast (2), and are transitioned through the hollow adapter at the flexible actuator torque area (221) or the cable winding ring.

4. The active interactive intelligent electronic eye system for disease and pest inspection according to claim 1, characterized in that, In view of the imaging defects caused by high-speed motion and occlusion of living insects, the system is also used for: Event-driven acquisition: a high-frame-rate near-infrared channel composed of the infrared camera (4) and the active near-infrared illumination ring (5) is used to detect motion saliency in real time, and combined with instance segmentation and lesion quantification results, an evidence mode is triggered according to the body segment proportion criterion output by the pre-trained skeleton network model; Multi-view geometry aggregation: the gimbal mast (2) is controlled to quickly switch small-baseline view poses according to the view domain analysis results, and multi-frame alignment is performed through depth guidance registration; Physical consistent reconstruction: in the computing unit (8), geometric normalization processing and time domain micro-motion amplification algorithm are fused to perform joint deblurring and super-resolution reconstruction; Closed-loop forensics: when occlusion causes the clarity to be lower than a second set threshold, the mechanical arm (6) and the flexible actuator (7) are dispatched to flip and resample, and an evidence package with three-dimensional scale and spatiotemporal label is output.

5. The active interactive pest and disease inspection intelligent electronic eye system according to claim 1, characterized in that, the computing unit (8) is used for three-dimensional reconstruction and pose planning, geometric normalization image processing, clear image reconstruction, instance segmentation, and pest / disease identification and quantification, and is used for edge computing and learning update; the computing unit (8) performs angle-distance normalization processing on the infrared intensity based on the leaf normal and camera-target distance obtained by depth estimation, and triggers active resampling when the visibility is lower than a first set threshold; the computing unit (8) constructs a point cloud / grid and performs view domain analysis, performs coverage measurement and blind area analysis, and generates a 360° pose planning path for main patrol and supplementary shooting; the geometric normalization includes normalization and highlight suppression processing on the infrared intensity according to the leaf normal, camera-target distance, and active illumination intensity; depth registration, time domain filtering, and micro-motion amplification are performed on the short-time sequence images obtained after the leaf flipping / turning operation, so as to enhance the detection sensitivity of insect micro-motion features; the instance segmentation outputs a single leaf / single fruit mask and its normal in three-dimensional space to obtain curved surface area, lesion area, and severity score; the system supports online incremental learning, which is used to update the identification model parameters on the edge according to a small amount of labeled samples, and the system can integrate a micro-quantity fixed-point spraying unit to implement precise pesticide application or labeling on the positioned lesion area; an environment map is constructed based on depth data, and coverage / blind area evaluation is performed to generate a 360° main patrol and supplementary shooting path; the leaf normal and camera-target distance are estimated by depth estimation, and the infrared image is geometrically normalized; instance segmentation is performed in three-dimensional space and visibility is evaluated; when the leaf back visibility is lower than a first set threshold, the mechanical arm (6) and the flexible actuator (7) are controlled to flip / turn the leaves and resample; pest / disease identification and quantification are performed on the normalized infrared / depth fusion features and geographical labeling is given; closed-loop supplementary shooting and optional fixed-point disposal are triggered according to the blind area statistical results; Detecting the motion saliency of the insect body through a high-frame-rate near-infrared channel, and starting an evidence collection mode when the confidence exceeds a third set threshold; generating a collection path of at least three small-baseline visual positions based on point cloud visual field analysis; reprojecting multi-angle frames to a unified coordinate system by using depth-infrared registration external parameters, and performing geometric normalization and highlight suppression; reconstructing high-definition images of insects by combining time-domain deblurring and multi-frame super-resolution reconstruction technologies; When the clarity is lower than a second set threshold due to leaf edge occlusion, controlling the flexible actuator (7) to perform a leaf turning action and re-collection; Enhancing the texture features of the insect body antennae and body segments based on micro-motion amplification; Outputting an evidence package containing three-dimensional scale estimation, spatiotemporal position labels, and hash signatures; The model in the system takes multi-modal input images as the starting point, normalizes the size of the input images, and records the mapping parameters; According to the leaf normal vector estimated by the depth field, the camera-target distance, and the lighting / exposure information, perform angle-distance-lighting geometric normalization on the infrared channel to obtain an intensity map independent of the viewing angle; perform weather / visibility discrimination and restoration on the normalized image to obtain an image for subsequent analysis; construct multi-channel edge and texture lightweight features and fuse them with temperature / reflection channels in a multi-branch encoder to generate a feature representation for detection / segmentation; the target detection / instance segmentation module based on the feature representation outputs instance regions and confidence, and calculates quantitative indicators under three-dimensional / geometric constraints, while performing imaging quality and uncertainty evaluation; when the results meet the first preset condition, output the segmentation and quantitative results, and when the first preset condition is not met, return instructions to the pose planning / active re-collection module to trigger gimbal fine tuning or leaf re-collection.

6. A method based on the active interactive pest patrol intelligent electronic eye system according to any one of claims 1 to 5, characterized in that, The method comprises: Step 1. Device installation and initialization; Step 1.1 Mechanism hierarchy and installation: arranging a body (9) on a mobile platform chassis (1), installing a mechanical arm (6) and a computing unit (8) at both ends of the body (9), fixing a depth camera (3), an infrared camera (4), an active near-infrared lighting ring (5), and an anti-glare shade (11) on a camera fixing area (210), detachably connecting the mechanical arm (6) to a flexible actuator fixing area (220) through a flexible actuator torque area (221), and installing a flexible actuator (7) at the end of the mechanical arm (6); Step 1.2 Calibration and synchronization: completing the internal and external parameter calibration and re-projection of the depth camera (3) and the infrared camera (4) in the computing unit (8), establishing a point cloud coordinate system and a consistent coordinate system for infrared intensity, and synchronizing the timestamps of the depth camera (3), the infrared camera (4), the active near-infrared lighting ring (5), and the communication and positioning unit (10), and recording the mapping parameters from pixels to physical dimensions; Step 1.3 Stabilization and lighting initialization: the gimbal mast (2) is turned on for stabilization control, and the base plate pitch / roll disturbance is decoupled; the active near-infrared lighting ring (5) is set to oblique incidence low-contrast mode, and the anti-reflection shade (11) is used to suppress fruit wax / dew bead highlights, providing high signal-to-noise ratio input for subsequent fruit surface texture and distance field acquisition; Step 2. Main patrol collection and quality control; A 360° main patrol pose set is generated along the fruit tree rows, and the mobile platform base plate (1) and the gimbal mast (2) are cooperated to the target pose for collecting depth D and infrared / near-infrared images I; online clarity, overexposure / underexposure, reflection, raindrop obstruction quality control is performed, and image frames that do not meet the second preset condition are immediately re-sampled to form a multi-view pose image group and a pose trajectory of the fruit; Step 3. Geometric normalization, registration and surface reconstruction; Based on the depth field, the distance and normal of the fruit surface to the camera are estimated, and the angle-distance-illumination geometric normalization is performed on the infrared intensity to weaken the radiation distortion caused by the viewing angle / distance; pixel-level registration and point cloud-intensity fusion are performed using the calibration results of step 1.2; the instance segmentation backbone network is called, and the single fruit mask is output, and curvature adaptive surface reconstruction and normal consistency filtering are performed under its constraint, and fruit surface parameterization is completed. The development provides a scale-unified measurement domain for subsequent area and circumferential quantization; Step 4. Lesion identification and three-dimensional quantization; In the reconstructed surface / development domain, lesion candidates are refined, segmented and artifact suppressed, and lesion pixel-level masks are output and back-projected to the real surface to obtain the real surface area and area ratio of the lesion; combined with the circumferential angle distribution and radiation difference, the maximum feature size and circumferential unevenness index are calculated to generate a fruit-oriented lesion quantization report; while retaining the pose, timestamp and calibrated version with each quantization result, it is used for evidence closed loop and traceability; Step 5. Blind area identification, supplementary sampling closed loop and result delivery; In the local grid / voxel map, the visible coverage volume is maintained, the visibility of the target fruit is estimated according to the field of view, obstruction and normal, and at least the inside of the fruit table / branch obstruction area is marked as a blind area and triggers secondary pose re-planning to obtain redundant imaging around multiple baseline viewing angles; when the network connectivity does not meet the third preset condition, the interpretation and evidence package generation are preferentially completed at the edge, and the original and derived elements are uploaded to the cloud platform in an intermittent and synchronous manner, and the host computer displays the lesion heat map and quantization index of the current plant / current row in real time to guide flower and fruit protection, thinning or disease plant marking and disposal, realizing stable, comparable and traceable three-dimensional quantization of fruit lesions in the "weak network-strong obstruction-high reflection" scene.

7. The method of the active interactive pest and disease patrol intelligent electronic eye system according to claim 6, characterized in that, The 360° master pose set in step 2 specifically includes the following steps: First, self-localization and mapping are completed by fusion of laser / visual odometry and global satellite navigation system. Time synchronization is performed on depth camera (3), infrared camera (4) and active near-infrared illumination ring (5) and internal and external hand-eye calibration parameters are loaded. A 360° master pose set around the target plant is generated in the reachable space of the robotic arm and gimbal and multimodal data are collected in sequence. Online quality control of sharpness, overexposure and raindrop occlusion is performed. If the second preset is not met, the quality control is not performed. The image frames under the conditions are automatically re-sampling; the computing unit (8) estimates the leaf normal and camera-target distance based on the depth, performs angle-distance-illumination intensity driven geometric normalization and specular suppression on the infrared image, and performs non-uniformity correction / emissivity correction and dehazing / denoising clear image reconstruction; the multi-pose depth frames are fused to obtain local point cloud / mesh, and visual field analysis is carried out to obtain coverage and blind zone distribution, and under three-dimensional constraints, the detection and instance segmentation of leaves / fruits / stems and suspicious insects, leaf front / back discrimination and visibility / uncertainty are completed. Qualitative assessment; when the visibility of the leaf underside is lower than the first set threshold or the identification uncertainty exceeds the tolerance, the system triggers closed-loop re-sampling: based on the blind zone distribution, the gimbal fine-tuning and robotic arm pose are solved, the contact point and contact direction are determined, and a virtual wall / safety boundary is set; the flexible actuator (7) completes leaf turning according to the three-stage trajectory of approach-leaf-release under the coordination of the support force PID channel and the torque PID channel, and combines the friction map / contact surface monitoring to prevent damage. After completion, the leaf underside is quickly resampled under local supplementary lighting; the short time sequence of re-sampling is performed with depth registration, temporal filtering and micro-motion amplification, and insect detection and automatic tracking are realized by combining optical flow / trajectory temporal characteristics, and the lesion surface area, minimum distance to the main leaf vein and severity index are quantified on the reconstructed surface; the system generates evidence package from multimodal images, reconstructed grid, trajectory ID, pose and timestamp and writes it into the anchor point of simultaneous positioning and map building / global satellite navigation system; when the coverage reaches the fourth set threshold, the inspection of the plant ends, and when it does not reach the fourth set threshold, the re-sampling plan continues to form a "main inspection-assessment-re-sampling" closed loop.

Images